2024-03-04T17:26:24.653 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-04T17:26:24.653 DEBUG:teuthology.run:Teuthology command: teuthology --name suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi --description rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} --archive /home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581388 --verbose --owner scheduled_suriarte@teuthology -- /home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581388/orig.config.yaml 2024-03-04T17:26:24.701 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T17:26:24.788 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581388 branch: wip-rgw-d4n description: rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} email: null first_in_suite: false job_id: '7581388' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rgw-d4n ceph: conf: client: debug rgw: 20 rgw bucket counters cache: true rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw torrent flag: true rgw user counters cache: true setgroup: ceph setuser: ceph global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled - \(OSD_SLOW_PING_TIME sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 openssl_keys: rgw.client.0: ca: root client: client.0 embed-key: true root: client: client.0 cn: teuthology install: - client.0 key-type: rsa:4096 rgw: client.0: ssl certificate: rgw.client.0 valgrind: - --tool=memcheck - --max-threads=1024 compression type: random ec-data-pool: true erasure_code_profile: crush-failure-domain: osd k: 3 m: 1 name: testprofile frontend: beast storage classes: LUKEWARM, FROZEN s3tests: force-branch: ceph-master slow_backend: true storage classes: LUKEWARM, FROZEN thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rgw-d4n sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 owner: scheduled_suriarte@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 sleep_before_teardown: 0 suite: rgw suite_branch: wip-rgw-d4n suite_path: /home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa suite_relpath: qa suite_repo: https://github.com/pritha-srivastava/ceph.git suite_sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0JC0xWUe9ufW+3yiztg7AXm5cz+sCYYvKOzGtg6sWZrVjG/ebql9HW6jJBru2iJ0jgDcg78ZnAhySMaXyig/c= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEyUHbYY+aKXXfOMbBl5Pv6EAyHHpgEvugTsX2szT1eNtIJZAZMkVOWzyqrYvW9HYqqa2VMo9BLs1tuZEe3Moo= tasks: - install: extra_system_packages: deb: - s3cmd - maven rpm: - s3cmd - maven - ceph: null - openssl_keys: null - rgw: client.0: null - tox: - client.0 - workunit: clients: client.0: - rgw/run-bucket-check.sh - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - cls/test_cls_rgw_gc.sh - cls/test_cls_rgw_stats.sh - cls/test_cls_cmpomap.sh - cls/test_cls_2pc_queue.sh - rgw/test_rgw_gc_log.sh - rgw/test_rgw_obj.sh - rgw/test_librgw_file.sh - rgw/test_awssdkv4_sig.sh - workunit: clients: client.0: - rgw/test_rgw_s3_mp_reupload.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - workunit: clients: client.0: - rgw/run-reshard.sh - s3tests-java: client.0: force-branch: ceph-master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: rgw_server: client.0 - workunit: clients: client.0: - rgw/run-versioning.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-03-04_17:13:34 tube: smithi user: suriarte verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-04T17:26:24.789 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa; will attempt to use it 2024-03-04T17:26:24.789 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks 2024-03-04T17:26:24.790 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-04T17:26:24.791 INFO:teuthology.task.internal:Checking packages... 2024-03-04T17:26:24.816 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '3aec390336fdfdc8f927818b718cfb0361bd47c8' 2024-03-04T17:26:24.816 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T17:26:24.816 INFO:teuthology.packaging:ref: None 2024-03-04T17:26:24.816 INFO:teuthology.packaging:tag: None 2024-03-04T17:26:24.816 INFO:teuthology.packaging:branch: wip-rgw-d4n 2024-03-04T17:26:24.816 INFO:teuthology.packaging:sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:26:24.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rgw-d4n 2024-03-04T17:26:24.972 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:26:24.974 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-04T17:26:24.980 INFO:teuthology.task.internal:no buildpackages task found 2024-03-04T17:26:24.980 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-04T17:26:25.019 INFO:teuthology.task.internal:Saving configuration 2024-03-04T17:26:25.038 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-04T17:26:25.043 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T17:26:25.067 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581388', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 17:14:33.389814', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEUu7/Pz1sn06+xPlQOtTDS/qUZD1fVNZ4Ues2VZ+ljv'} 2024-03-04T17:26:25.092 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581388', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 17:14:33.391212', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICYrLjY9iGpZnXZtmYh6hfKgteA6/n8fuwG67YHwaQ2I'} 2024-03-04T17:26:25.093 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-04T17:26:25.098 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-03-04T17:26:25.098 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-03-04T17:26:25.098 INFO:teuthology.run_tasks:Running task console_log... 2024-03-04T17:26:25.193 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1da8a67dc0>, signals=[15]) 2024-03-04T17:26:25.194 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-04T17:26:25.200 INFO:teuthology.task.internal:Opening connections... 2024-03-04T17:26:25.200 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-03-04T17:26:25.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:26:25.294 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-03-04T17:26:25.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:26:25.376 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-04T17:26:25.383 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-03-04T17:26:25.389 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-03-04T17:26:25.389 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-03-04T17:26:25.434 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="Ubuntu" 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="22.04" 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_CODENAME=jammy 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:ID=ubuntu 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE=debian 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-04T17:26:25.435 INFO:teuthology.orchestra.run.smithi047.stdout:UBUNTU_CODENAME=jammy 2024-03-04T17:26:25.436 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-03-04T17:26:25.459 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-03-04T17:26:25.464 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-03-04T17:26:25.465 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="Ubuntu" 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="22.04" 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_CODENAME=jammy 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:ID=ubuntu 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE=debian 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-04T17:26:25.510 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-04T17:26:25.511 INFO:teuthology.orchestra.run.smithi152.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-04T17:26:25.511 INFO:teuthology.orchestra.run.smithi152.stdout:UBUNTU_CODENAME=jammy 2024-03-04T17:26:25.511 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-03-04T17:26:25.534 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-04T17:26:25.544 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-04T17:26:25.596 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-04T17:26:25.596 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-03-04T17:26:25.599 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-03-04T17:26:25.605 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-04T17:26:25.612 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-04T17:26:25.612 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-03-04T17:26:25.648 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-03-04T17:26:25.664 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-04T17:26:25.719 INFO:teuthology.run_tasks:Running task kernel... 2024-03-04T17:26:25.734 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-04T17:26:25.734 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-04T17:26:25.734 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T17:26:25.735 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T17:26:25.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:26:25.739 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-03-04T17:26:25.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:26:25.740 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-03-04T17:26:25.786 INFO:teuthology.orchestra.run.smithi152.stdout:5.15.0-56-generic 2024-03-04T17:26:25.786 INFO:teuthology.task.kernel:Running kernel on smithi152: 5.15.0-56-generic 2024-03-04T17:26:25.786 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-get clean 2024-03-04T17:26:25.787 INFO:teuthology.orchestra.run.smithi047.stdout:5.15.0-56-generic 2024-03-04T17:26:25.787 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.15.0-56-generic 2024-03-04T17:26:25.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get clean 2024-03-04T17:26:25.923 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get update 2024-03-04T17:26:25.933 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-get update 2024-03-04T17:26:26.160 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:26:26.161 INFO:teuthology.orchestra.run.smithi152.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-04T17:26:26.191 INFO:teuthology.orchestra.run.smithi152.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-04T17:26:26.214 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:26:26.214 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-04T17:26:26.277 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-04T17:26:26.388 INFO:teuthology.orchestra.run.smithi152.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-04T17:26:26.452 INFO:teuthology.orchestra.run.smithi152.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [413 kB] 2024-03-04T17:26:26.511 INFO:teuthology.orchestra.run.smithi152.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,422 kB] 2024-03-04T17:26:26.640 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-04T17:26:26.645 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [413 kB] 2024-03-04T17:26:26.776 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,422 kB] 2024-03-04T17:26:26.859 INFO:teuthology.orchestra.run.smithi152.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,205 kB] 2024-03-04T17:26:26.929 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,205 kB] 2024-03-04T17:26:26.951 INFO:teuthology.orchestra.run.smithi152.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [579 kB] 2024-03-04T17:26:27.075 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [579 kB] 2024-03-04T17:26:27.103 INFO:teuthology.orchestra.run.smithi152.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [279 kB] 2024-03-04T17:26:27.112 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [219 kB] 2024-03-04T17:26:27.117 INFO:teuthology.orchestra.run.smithi047.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [279 kB] 2024-03-04T17:26:27.127 INFO:teuthology.orchestra.run.smithi047.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-04T17:26:27.128 INFO:teuthology.orchestra.run.smithi047.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-04T17:26:27.129 INFO:teuthology.orchestra.run.smithi047.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,476 kB] 2024-03-04T17:26:27.131 INFO:teuthology.orchestra.run.smithi047.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-04T17:26:27.132 INFO:teuthology.orchestra.run.smithi047.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-04T17:26:27.134 INFO:teuthology.orchestra.run.smithi047.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,504 kB] 2024-03-04T17:26:27.160 INFO:teuthology.orchestra.run.smithi152.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-04T17:26:27.161 INFO:teuthology.orchestra.run.smithi152.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,504 kB] 2024-03-04T17:26:27.176 INFO:teuthology.orchestra.run.smithi152.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [219 kB] 2024-03-04T17:26:27.180 INFO:teuthology.orchestra.run.smithi152.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-04T17:26:27.180 INFO:teuthology.orchestra.run.smithi152.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-04T17:26:27.182 INFO:teuthology.orchestra.run.smithi152.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,476 kB] 2024-03-04T17:26:27.235 INFO:teuthology.orchestra.run.smithi047.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [247 kB] 2024-03-04T17:26:27.235 INFO:teuthology.orchestra.run.smithi047.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [244 kB] 2024-03-04T17:26:27.240 INFO:teuthology.orchestra.run.smithi047.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T17:26:27.240 INFO:teuthology.orchestra.run.smithi047.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T17:26:27.240 INFO:teuthology.orchestra.run.smithi047.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [846 kB] 2024-03-04T17:26:27.256 INFO:teuthology.orchestra.run.smithi047.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [596 kB] 2024-03-04T17:26:27.259 INFO:teuthology.orchestra.run.smithi152.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-04T17:26:27.260 INFO:teuthology.orchestra.run.smithi152.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [247 kB] 2024-03-04T17:26:27.327 INFO:teuthology.orchestra.run.smithi152.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T17:26:27.328 INFO:teuthology.orchestra.run.smithi152.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,052 kB] 2024-03-04T17:26:27.351 INFO:teuthology.orchestra.run.smithi152.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [244 kB] 2024-03-04T17:26:27.353 INFO:teuthology.orchestra.run.smithi152.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [691 kB] 2024-03-04T17:26:27.355 INFO:teuthology.orchestra.run.smithi152.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T17:26:27.355 INFO:teuthology.orchestra.run.smithi152.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [596 kB] 2024-03-04T17:26:27.366 INFO:teuthology.orchestra.run.smithi152.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [237 kB] 2024-03-04T17:26:27.366 INFO:teuthology.orchestra.run.smithi152.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [846 kB] 2024-03-04T17:26:27.370 INFO:teuthology.orchestra.run.smithi152.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-04T17:26:27.371 INFO:teuthology.orchestra.run.smithi152.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-04T17:26:27.371 INFO:teuthology.orchestra.run.smithi152.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-04T17:26:27.372 INFO:teuthology.orchestra.run.smithi152.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-04T17:26:27.373 INFO:teuthology.orchestra.run.smithi152.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-04T17:26:27.373 INFO:teuthology.orchestra.run.smithi152.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-03-04T17:26:27.374 INFO:teuthology.orchestra.run.smithi152.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-03-04T17:26:27.381 INFO:teuthology.orchestra.run.smithi152.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [161 kB] 2024-03-04T17:26:27.384 INFO:teuthology.orchestra.run.smithi152.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-04T17:26:27.384 INFO:teuthology.orchestra.run.smithi152.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-04T17:26:27.385 INFO:teuthology.orchestra.run.smithi152.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-04T17:26:27.385 INFO:teuthology.orchestra.run.smithi152.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-04T17:26:27.386 INFO:teuthology.orchestra.run.smithi152.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-04T17:26:27.388 INFO:teuthology.orchestra.run.smithi152.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-03-04T17:26:27.404 INFO:teuthology.orchestra.run.smithi047.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,052 kB] 2024-03-04T17:26:27.408 INFO:teuthology.orchestra.run.smithi152.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-04T17:26:27.408 INFO:teuthology.orchestra.run.smithi152.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-03-04T17:26:27.409 INFO:teuthology.orchestra.run.smithi152.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2024-03-04T17:26:27.409 INFO:teuthology.orchestra.run.smithi152.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-03-04T17:26:27.410 INFO:teuthology.orchestra.run.smithi152.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-04T17:26:27.438 INFO:teuthology.orchestra.run.smithi047.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [161 kB] 2024-03-04T17:26:27.835 INFO:teuthology.orchestra.run.smithi047.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-04T17:26:27.841 INFO:teuthology.orchestra.run.smithi047.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-04T17:26:27.841 INFO:teuthology.orchestra.run.smithi047.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-04T17:26:27.856 INFO:teuthology.orchestra.run.smithi047.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-04T17:26:27.858 INFO:teuthology.orchestra.run.smithi047.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-04T17:26:28.042 INFO:teuthology.orchestra.run.smithi047.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [691 kB] 2024-03-04T17:26:28.081 INFO:teuthology.orchestra.run.smithi047.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [237 kB] 2024-03-04T17:26:28.095 INFO:teuthology.orchestra.run.smithi047.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-04T17:26:28.095 INFO:teuthology.orchestra.run.smithi047.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-04T17:26:28.099 INFO:teuthology.orchestra.run.smithi047.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-04T17:26:28.099 INFO:teuthology.orchestra.run.smithi047.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-04T17:26:28.100 INFO:teuthology.orchestra.run.smithi047.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-04T17:26:28.100 INFO:teuthology.orchestra.run.smithi047.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-03-04T17:26:28.101 INFO:teuthology.orchestra.run.smithi047.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-03-04T17:26:28.124 INFO:teuthology.orchestra.run.smithi047.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-03-04T17:26:28.289 INFO:teuthology.orchestra.run.smithi047.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-04T17:26:28.373 INFO:teuthology.orchestra.run.smithi047.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-03-04T17:26:28.374 INFO:teuthology.orchestra.run.smithi047.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2024-03-04T17:26:28.498 INFO:teuthology.orchestra.run.smithi047.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-03-04T17:26:28.552 INFO:teuthology.orchestra.run.smithi047.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-04T17:26:32.539 INFO:teuthology.orchestra.run.smithi152.stdout:Fetched 11.9 MB in 2s (5,309 kB/s) 2024-03-04T17:26:33.156 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 11.9 MB in 3s (4,680 kB/s) 2024-03-04T17:26:33.886 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2024-03-04T17:26:33.904 DEBUG:teuthology.orchestra.run.smithi152:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T17:26:33.962 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2024-03-04T17:26:34.148 INFO:teuthology.orchestra.run.smithi152.stdout:Building dependency tree... 2024-03-04T17:26:34.148 INFO:teuthology.orchestra.run.smithi152.stdout:Reading state information... 2024-03-04T17:26:34.310 INFO:teuthology.orchestra.run.smithi152.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:26:34.310 INFO:teuthology.orchestra.run.smithi152.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:26:34.310 INFO:teuthology.orchestra.run.smithi152.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:26:34.310 INFO:teuthology.orchestra.run.smithi152.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:26:34.310 INFO:teuthology.orchestra.run.smithi152.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-04T17:26:34.310 INFO:teuthology.orchestra.run.smithi152.stdout: python2.7-minimal 2024-03-04T17:26:34.310 INFO:teuthology.orchestra.run.smithi152.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:26:34.311 INFO:teuthology.orchestra.run.smithi152.stdout:The following additional packages will be installed: 2024-03-04T17:26:34.311 INFO:teuthology.orchestra.run.smithi152.stdout: linux-generic linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T17:26:34.311 INFO:teuthology.orchestra.run.smithi152.stdout: linux-headers-generic linux-image-5.15.0-97-generic 2024-03-04T17:26:34.311 INFO:teuthology.orchestra.run.smithi152.stdout: linux-modules-5.15.0-97-generic linux-modules-extra-5.15.0-97-generic 2024-03-04T17:26:34.312 INFO:teuthology.orchestra.run.smithi152.stdout:Suggested packages: 2024-03-04T17:26:34.313 INFO:teuthology.orchestra.run.smithi152.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-04T17:26:34.313 INFO:teuthology.orchestra.run.smithi152.stdout:Recommended packages: 2024-03-04T17:26:34.313 INFO:teuthology.orchestra.run.smithi152.stdout: thermald 2024-03-04T17:26:34.364 INFO:teuthology.orchestra.run.smithi152.stdout:The following NEW packages will be installed: 2024-03-04T17:26:34.365 INFO:teuthology.orchestra.run.smithi152.stdout: linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T17:26:34.365 INFO:teuthology.orchestra.run.smithi152.stdout: linux-image-5.15.0-97-generic linux-modules-5.15.0-97-generic 2024-03-04T17:26:34.365 INFO:teuthology.orchestra.run.smithi152.stdout: linux-modules-extra-5.15.0-97-generic 2024-03-04T17:26:34.365 INFO:teuthology.orchestra.run.smithi152.stdout:The following packages will be upgraded: 2024-03-04T17:26:34.366 INFO:teuthology.orchestra.run.smithi152.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-04T17:26:34.430 INFO:teuthology.orchestra.run.smithi152.stdout:3 upgraded, 5 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T17:26:34.430 INFO:teuthology.orchestra.run.smithi152.stdout:Need to get 113 MB of archives. 2024-03-04T17:26:34.430 INFO:teuthology.orchestra.run.smithi152.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-04T17:26:34.430 INFO:teuthology.orchestra.run.smithi152.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-97-generic amd64 5.15.0-97.107 [22.6 MB] 2024-03-04T17:26:34.675 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-03-04T17:26:34.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T17:26:34.758 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-03-04T17:26:34.964 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2024-03-04T17:26:34.964 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2024-03-04T17:26:35.140 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:26:35.141 INFO:teuthology.orchestra.run.smithi047.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:26:35.141 INFO:teuthology.orchestra.run.smithi047.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:26:35.141 INFO:teuthology.orchestra.run.smithi047.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:26:35.141 INFO:teuthology.orchestra.run.smithi047.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-04T17:26:35.142 INFO:teuthology.orchestra.run.smithi047.stdout: python2.7-minimal 2024-03-04T17:26:35.142 INFO:teuthology.orchestra.run.smithi047.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:26:35.143 INFO:teuthology.orchestra.run.smithi047.stdout:The following additional packages will be installed: 2024-03-04T17:26:35.143 INFO:teuthology.orchestra.run.smithi047.stdout: linux-generic linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T17:26:35.143 INFO:teuthology.orchestra.run.smithi047.stdout: linux-headers-generic linux-image-5.15.0-97-generic 2024-03-04T17:26:35.143 INFO:teuthology.orchestra.run.smithi047.stdout: linux-modules-5.15.0-97-generic linux-modules-extra-5.15.0-97-generic 2024-03-04T17:26:35.145 INFO:teuthology.orchestra.run.smithi047.stdout:Suggested packages: 2024-03-04T17:26:35.145 INFO:teuthology.orchestra.run.smithi047.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-04T17:26:35.145 INFO:teuthology.orchestra.run.smithi047.stdout:Recommended packages: 2024-03-04T17:26:35.145 INFO:teuthology.orchestra.run.smithi047.stdout: thermald 2024-03-04T17:26:35.152 INFO:teuthology.orchestra.run.smithi152.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-97-generic amd64 5.15.0-97.107 [11.5 MB] 2024-03-04T17:26:35.193 INFO:teuthology.orchestra.run.smithi047.stdout:The following NEW packages will be installed: 2024-03-04T17:26:35.193 INFO:teuthology.orchestra.run.smithi047.stdout: linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T17:26:35.194 INFO:teuthology.orchestra.run.smithi047.stdout: linux-image-5.15.0-97-generic linux-modules-5.15.0-97-generic 2024-03-04T17:26:35.194 INFO:teuthology.orchestra.run.smithi047.stdout: linux-modules-extra-5.15.0-97-generic 2024-03-04T17:26:35.195 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages will be upgraded: 2024-03-04T17:26:35.195 INFO:teuthology.orchestra.run.smithi047.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-04T17:26:35.314 INFO:teuthology.orchestra.run.smithi047.stdout:3 upgraded, 5 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T17:26:35.314 INFO:teuthology.orchestra.run.smithi047.stdout:Need to get 113 MB of archives. 2024-03-04T17:26:35.314 INFO:teuthology.orchestra.run.smithi047.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-04T17:26:35.314 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-97-generic amd64 5.15.0-97.107 [22.6 MB] 2024-03-04T17:26:35.395 INFO:teuthology.orchestra.run.smithi152.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-97-generic amd64 5.15.0-97.107 [63.8 MB] 2024-03-04T17:26:36.258 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-97-generic amd64 5.15.0-97.107 [11.5 MB] 2024-03-04T17:26:36.583 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-97-generic amd64 5.15.0-97.107 [63.8 MB] 2024-03-04T17:26:36.876 INFO:teuthology.orchestra.run.smithi152.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.97.92 [1,696 B] 2024-03-04T17:26:36.876 INFO:teuthology.orchestra.run.smithi152.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.97.92 [2,494 B] 2024-03-04T17:26:36.877 INFO:teuthology.orchestra.run.smithi152.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97 all 5.15.0-97.107 [12.3 MB] 2024-03-04T17:26:37.091 INFO:teuthology.orchestra.run.smithi152.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97-generic amd64 5.15.0-97.107 [2,892 kB] 2024-03-04T17:26:37.144 INFO:teuthology.orchestra.run.smithi152.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.97.92 [2,342 B] 2024-03-04T17:26:37.523 INFO:teuthology.orchestra.run.smithi152.stdout:Fetched 113 MB in 3s (40.8 MB/s) 2024-03-04T17:26:37.663 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.97.92 [1,696 B] 2024-03-04T17:26:37.663 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.97.92 [2,494 B] 2024-03-04T17:26:37.663 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97 all 5.15.0-97.107 [12.3 MB] 2024-03-04T17:26:37.688 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package linux-modules-5.15.0-97-generic. 2024-03-04T17:26:37.826 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97-generic amd64 5.15.0-97.107 [2,892 kB] 2024-03-04T17:26:37.860 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.97.92 [2,342 B] 2024-03-04T17:26:38.307 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 113 MB in 3s (42.6 MB/s) 2024-03-04T17:26:38.525 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-modules-5.15.0-97-generic. 2024-03-04T17:26:39.741 INFO:teuthology.orchestra.run.smithi152.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-04T17:26:39.745 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../0-linux-modules-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:26:39.875 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:40.536 INFO:teuthology.orchestra.run.smithi047.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-04T17:26:40.540 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../0-linux-modules-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:26:40.706 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:42.159 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package linux-image-5.15.0-97-generic. 2024-03-04T17:26:42.180 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../1-linux-image-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:26:42.271 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:42.754 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-97-generic. 2024-03-04T17:26:42.774 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:26:42.812 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:42.874 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-image-5.15.0-97-generic. 2024-03-04T17:26:42.895 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../1-linux-image-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:26:42.967 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-97-generic. 2024-03-04T17:26:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:26:43.503 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:49.726 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../3-linux-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:26:49.851 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking linux-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:26:50.184 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:26:50.305 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking linux-image-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:26:50.404 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../3-linux-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:26:50.615 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package linux-headers-5.15.0-97. 2024-03-04T17:26:50.627 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../5-linux-headers-5.15.0-97_5.15.0-97.107_all.deb ... 2024-03-04T17:26:50.665 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T17:26:50.890 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:26:51.018 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-image-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:26:51.379 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-headers-5.15.0-97. 2024-03-04T17:26:51.402 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../5-linux-headers-5.15.0-97_5.15.0-97.107_all.deb ... 2024-03-04T17:26:51.447 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T17:26:56.115 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package linux-headers-5.15.0-97-generic. 2024-03-04T17:26:56.151 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../6-linux-headers-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:26:56.192 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:57.395 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package linux-headers-5.15.0-97-generic. 2024-03-04T17:26:57.432 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../6-linux-headers-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:26:57.470 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:58.844 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:26:58.944 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking linux-headers-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:26:59.277 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T17:26:59.416 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:59.545 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up linux-headers-generic (5.15.0.97.92) ... 2024-03-04T17:26:59.671 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:27:00.925 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:27:01.026 INFO:teuthology.orchestra.run.smithi152.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:27:01.026 INFO:teuthology.orchestra.run.smithi152.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-04T17:27:01.027 INFO:teuthology.orchestra.run.smithi152.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:27:01.027 INFO:teuthology.orchestra.run.smithi152.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-97-generic 2024-03-04T17:27:01.072 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking linux-headers-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:27:01.207 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:27:01.441 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T17:27:02.199 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:27:02.447 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-headers-generic (5.15.0.97.92) ... 2024-03-04T17:27:02.609 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:27:02.658 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:27:03.960 INFO:teuthology.orchestra.run.smithi047.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:27:03.960 INFO:teuthology.orchestra.run.smithi047.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-04T17:27:03.960 INFO:teuthology.orchestra.run.smithi047.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:27:03.960 INFO:teuthology.orchestra.run.smithi047.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-97-generic 2024-03-04T17:27:04.245 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:27:04.552 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up linux-image-generic (5.15.0.97.92) ... 2024-03-04T17:27:04.661 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up linux-generic (5.15.0.97.92) ... 2024-03-04T17:27:04.779 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:27:04.846 INFO:teuthology.orchestra.run.smithi152.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-04T17:27:04.846 INFO:teuthology.orchestra.run.smithi152.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:27:05.713 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:27:07.177 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-image-generic (5.15.0.97.92) ... 2024-03-04T17:27:07.282 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up linux-generic (5.15.0.97.92) ... 2024-03-04T17:27:07.383 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:27:08.085 INFO:teuthology.orchestra.run.smithi047.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-04T17:27:08.085 INFO:teuthology.orchestra.run.smithi047.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:27:19.066 INFO:teuthology.orchestra.run.smithi152.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-04T17:27:19.066 INFO:teuthology.orchestra.run.smithi152.stdout:Sourcing file `/etc/default/grub' 2024-03-04T17:27:19.100 INFO:teuthology.orchestra.run.smithi152.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T17:27:19.170 INFO:teuthology.orchestra.run.smithi152.stdout:Generating grub configuration file ... 2024-03-04T17:27:19.479 INFO:teuthology.orchestra.run.smithi152.stdout:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:27:19.493 INFO:teuthology.orchestra.run.smithi152.stdout:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:27:19.879 INFO:teuthology.orchestra.run.smithi152.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:27:19.885 INFO:teuthology.orchestra.run.smithi152.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T17:27:19.957 INFO:teuthology.orchestra.run.smithi152.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T17:27:19.964 INFO:teuthology.orchestra.run.smithi152.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T17:27:20.122 INFO:teuthology.orchestra.run.smithi152.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T17:27:20.122 INFO:teuthology.orchestra.run.smithi152.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T17:27:20.122 INFO:teuthology.orchestra.run.smithi152.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T17:27:20.173 INFO:teuthology.orchestra.run.smithi152.stdout:done 2024-03-04T17:27:20.626 DEBUG:teuthology.orchestra.run.smithi152:> dpkg -s linux-image-generic 2024-03-04T17:27:20.650 INFO:teuthology.orchestra.run.smithi152.stdout:Package: linux-image-generic 2024-03-04T17:27:20.650 INFO:teuthology.orchestra.run.smithi152.stdout:Status: install ok installed 2024-03-04T17:27:20.650 INFO:teuthology.orchestra.run.smithi152.stdout:Priority: optional 2024-03-04T17:27:20.651 INFO:teuthology.orchestra.run.smithi152.stdout:Section: kernel 2024-03-04T17:27:20.651 INFO:teuthology.orchestra.run.smithi152.stdout:Installed-Size: 21 2024-03-04T17:27:20.651 INFO:teuthology.orchestra.run.smithi152.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T17:27:20.651 INFO:teuthology.orchestra.run.smithi152.stdout:Architecture: amd64 2024-03-04T17:27:20.651 INFO:teuthology.orchestra.run.smithi152.stdout:Source: linux-meta 2024-03-04T17:27:20.651 INFO:teuthology.orchestra.run.smithi152.stdout:Version: 5.15.0.97.92 2024-03-04T17:27:20.651 INFO:teuthology.orchestra.run.smithi152.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T17:27:20.651 INFO:teuthology.orchestra.run.smithi152.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T17:27:20.651 INFO:teuthology.orchestra.run.smithi152.stdout:Recommends: thermald 2024-03-04T17:27:20.652 INFO:teuthology.orchestra.run.smithi152.stdout:Description: Generic Linux kernel image 2024-03-04T17:27:20.652 INFO:teuthology.orchestra.run.smithi152.stdout: This package will always depend on the latest generic kernel image 2024-03-04T17:27:20.652 INFO:teuthology.orchestra.run.smithi152.stdout: available. 2024-03-04T17:27:20.652 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-97-generic 2024-03-04T17:27:20.652 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T17:27:20.653 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T17:27:20.653 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-04T17:27:20.653 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi152.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T17:27:20.653 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-get clean 2024-03-04T17:27:20.851 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-get update 2024-03-04T17:27:20.984 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T17:27:20.984 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:27:20.999 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T17:27:21.020 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T17:27:22.304 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2024-03-04T17:27:22.321 DEBUG:teuthology.orchestra.run.smithi152:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T17:27:22.384 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2024-03-04T17:27:22.574 INFO:teuthology.orchestra.run.smithi152.stdout:Building dependency tree... 2024-03-04T17:27:22.575 INFO:teuthology.orchestra.run.smithi152.stdout:Reading state information... 2024-03-04T17:27:22.746 INFO:teuthology.orchestra.run.smithi152.stdout:linux-image-generic is already the newest version (5.15.0.97.92). 2024-03-04T17:27:22.746 INFO:teuthology.orchestra.run.smithi152.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:27:22.747 INFO:teuthology.orchestra.run.smithi152.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:27:22.747 INFO:teuthology.orchestra.run.smithi152.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:27:22.747 INFO:teuthology.orchestra.run.smithi152.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:27:22.747 INFO:teuthology.orchestra.run.smithi152.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:27:22.747 INFO:teuthology.orchestra.run.smithi152.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:27:22.747 INFO:teuthology.orchestra.run.smithi152.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:27:22.788 INFO:teuthology.orchestra.run.smithi152.stdout:0 upgraded, 0 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T17:27:22.790 DEBUG:teuthology.orchestra.run.smithi152:> dpkg -s linux-image-generic 2024-03-04T17:27:22.806 INFO:teuthology.orchestra.run.smithi152.stdout:Package: linux-image-generic 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Status: install ok installed 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Priority: optional 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Section: kernel 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Installed-Size: 21 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Architecture: amd64 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Source: linux-meta 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Version: 5.15.0.97.92 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Recommends: thermald 2024-03-04T17:27:22.807 INFO:teuthology.orchestra.run.smithi152.stdout:Description: Generic Linux kernel image 2024-03-04T17:27:22.808 INFO:teuthology.orchestra.run.smithi152.stdout: This package will always depend on the latest generic kernel image 2024-03-04T17:27:22.808 INFO:teuthology.orchestra.run.smithi152.stdout: available. 2024-03-04T17:27:22.808 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2024-03-04T17:27:22.853 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.e3LTfSajc6 2024-03-04T17:27:22.854 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /boot/grub/grub.cfg /tmp/tmp.e3LTfSajc6 2024-03-04T17:27:22.908 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.e3LTfSajc6 2024-03-04T17:27:23.026 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.e3LTfSajc6 is 10KB 2024-03-04T17:27:23.044 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.e3LTfSajc6 2024-03-04T17:27:23.049 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-04T17:27:23.069 INFO:teuthology.orchestra.run.smithi047.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-04T17:27:23.069 INFO:teuthology.orchestra.run.smithi047.stdout:Sourcing file `/etc/default/grub' 2024-03-04T17:27:23.101 INFO:teuthology.orchestra.run.smithi047.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T17:27:23.103 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:27:23.103 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-04T17:27:23.103 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-04T17:27:23.122 INFO:teuthology.orchestra.run.smithi047.stdout:Generating grub configuration file ... 2024-03-04T17:27:23.171 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-97-generic 2024-03-04T17:27:23.171 DEBUG:teuthology.orchestra.run.smithi152:> sudo update-grub 2024-03-04T17:27:23.459 INFO:teuthology.orchestra.run.smithi047.stdout:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:27:23.490 INFO:teuthology.orchestra.run.smithi047.stdout:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:27:23.838 INFO:teuthology.orchestra.run.smithi047.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:27:23.844 INFO:teuthology.orchestra.run.smithi047.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T17:27:23.915 INFO:teuthology.orchestra.run.smithi047.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T17:27:23.921 INFO:teuthology.orchestra.run.smithi047.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T17:27:24.077 INFO:teuthology.orchestra.run.smithi047.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T17:27:24.077 INFO:teuthology.orchestra.run.smithi047.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T17:27:24.077 INFO:teuthology.orchestra.run.smithi047.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T17:27:24.124 INFO:teuthology.orchestra.run.smithi047.stdout:done 2024-03-04T17:27:24.443 INFO:teuthology.orchestra.run.smithi152.stderr:Sourcing file `/etc/default/grub' 2024-03-04T17:27:24.445 INFO:teuthology.orchestra.run.smithi152.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T17:27:24.446 INFO:teuthology.orchestra.run.smithi152.stderr:Generating grub configuration file ... 2024-03-04T17:27:24.649 DEBUG:teuthology.orchestra.run.smithi047:> dpkg -s linux-image-generic 2024-03-04T17:27:24.684 INFO:teuthology.orchestra.run.smithi047.stdout:Package: linux-image-generic 2024-03-04T17:27:24.684 INFO:teuthology.orchestra.run.smithi047.stdout:Status: install ok installed 2024-03-04T17:27:24.684 INFO:teuthology.orchestra.run.smithi047.stdout:Priority: optional 2024-03-04T17:27:24.684 INFO:teuthology.orchestra.run.smithi047.stdout:Section: kernel 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout:Installed-Size: 21 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: amd64 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout:Source: linux-meta 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout:Version: 5.15.0.97.92 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout:Recommends: thermald 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout:Description: Generic Linux kernel image 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout: This package will always depend on the latest generic kernel image 2024-03-04T17:27:24.685 INFO:teuthology.orchestra.run.smithi047.stdout: available. 2024-03-04T17:27:24.685 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-97-generic 2024-03-04T17:27:24.686 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T17:27:24.686 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T17:27:24.686 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-04T17:27:24.686 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T17:27:24.686 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get clean 2024-03-04T17:27:24.755 INFO:teuthology.orchestra.run.smithi152.stderr:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:27:24.761 INFO:teuthology.orchestra.run.smithi152.stderr:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:27:24.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get update 2024-03-04T17:27:25.057 INFO:teuthology.orchestra.run.smithi152.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:27:25.063 INFO:teuthology.orchestra.run.smithi152.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T17:27:25.098 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:27:25.098 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T17:27:25.106 INFO:teuthology.orchestra.run.smithi152.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T17:27:25.109 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T17:27:25.111 INFO:teuthology.orchestra.run.smithi152.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T17:27:25.130 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T17:27:25.215 INFO:teuthology.orchestra.run.smithi152.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T17:27:25.215 INFO:teuthology.orchestra.run.smithi152.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T17:27:25.215 INFO:teuthology.orchestra.run.smithi152.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T17:27:25.236 INFO:teuthology.orchestra.run.smithi152.stderr:done 2024-03-04T17:27:25.239 DEBUG:teuthology.orchestra.run.smithi152:> sudo shutdown -r now 2024-03-04T17:27:26.622 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-03-04T17:27:26.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T17:27:26.702 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-03-04T17:27:26.901 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2024-03-04T17:27:26.901 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2024-03-04T17:27:27.080 INFO:teuthology.orchestra.run.smithi047.stdout:linux-image-generic is already the newest version (5.15.0.97.92). 2024-03-04T17:27:27.080 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:27:27.080 INFO:teuthology.orchestra.run.smithi047.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:27:27.080 INFO:teuthology.orchestra.run.smithi047.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:27:27.080 INFO:teuthology.orchestra.run.smithi047.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:27:27.081 INFO:teuthology.orchestra.run.smithi047.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:27:27.081 INFO:teuthology.orchestra.run.smithi047.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:27:27.081 INFO:teuthology.orchestra.run.smithi047.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:27:27.125 INFO:teuthology.orchestra.run.smithi047.stdout:0 upgraded, 0 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T17:27:27.126 DEBUG:teuthology.orchestra.run.smithi047:> dpkg -s linux-image-generic 2024-03-04T17:27:27.147 INFO:teuthology.orchestra.run.smithi047.stdout:Package: linux-image-generic 2024-03-04T17:27:27.147 INFO:teuthology.orchestra.run.smithi047.stdout:Status: install ok installed 2024-03-04T17:27:27.147 INFO:teuthology.orchestra.run.smithi047.stdout:Priority: optional 2024-03-04T17:27:27.147 INFO:teuthology.orchestra.run.smithi047.stdout:Section: kernel 2024-03-04T17:27:27.147 INFO:teuthology.orchestra.run.smithi047.stdout:Installed-Size: 21 2024-03-04T17:27:27.147 INFO:teuthology.orchestra.run.smithi047.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T17:27:27.147 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: amd64 2024-03-04T17:27:27.148 INFO:teuthology.orchestra.run.smithi047.stdout:Source: linux-meta 2024-03-04T17:27:27.148 INFO:teuthology.orchestra.run.smithi047.stdout:Version: 5.15.0.97.92 2024-03-04T17:27:27.148 INFO:teuthology.orchestra.run.smithi047.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T17:27:27.148 INFO:teuthology.orchestra.run.smithi047.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T17:27:27.148 INFO:teuthology.orchestra.run.smithi047.stdout:Recommends: thermald 2024-03-04T17:27:27.148 INFO:teuthology.orchestra.run.smithi047.stdout:Description: Generic Linux kernel image 2024-03-04T17:27:27.148 INFO:teuthology.orchestra.run.smithi047.stdout: This package will always depend on the latest generic kernel image 2024-03-04T17:27:27.148 INFO:teuthology.orchestra.run.smithi047.stdout: available. 2024-03-04T17:27:27.148 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-03-04T17:27:27.196 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.lE53MGYuyi 2024-03-04T17:27:27.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub/grub.cfg /tmp/tmp.lE53MGYuyi 2024-03-04T17:27:27.257 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.lE53MGYuyi 2024-03-04T17:27:27.380 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.lE53MGYuyi is 10KB 2024-03-04T17:27:27.401 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.lE53MGYuyi 2024-03-04T17:27:27.407 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-04T17:27:27.466 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:27:27.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-04T17:27:27.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-04T17:27:27.536 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-97-generic 2024-03-04T17:27:27.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo update-grub 2024-03-04T17:27:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:Sourcing file `/etc/default/grub' 2024-03-04T17:27:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T17:27:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-03-04T17:27:29.184 INFO:teuthology.orchestra.run.smithi047.stderr:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:27:29.191 INFO:teuthology.orchestra.run.smithi047.stderr:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:27:29.470 INFO:teuthology.orchestra.run.smithi047.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:27:29.476 INFO:teuthology.orchestra.run.smithi047.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T17:27:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T17:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T17:27:29.654 INFO:teuthology.orchestra.run.smithi047.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T17:27:29.654 INFO:teuthology.orchestra.run.smithi047.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T17:27:29.654 INFO:teuthology.orchestra.run.smithi047.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T17:27:29.664 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-03-04T17:27:29.665 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-03-04T17:27:55.266 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-04T17:27:55.267 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-04T17:27:55.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:27:59.720 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-04T17:27:59.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-04T17:27:59.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:28:29.761 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-03-04T17:28:38.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-04T17:28:38.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:28:57.152 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-03-04T17:28:59.724 DEBUG:teuthology.orchestra.remote:timed out 2024-03-04T17:29:08.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-04T17:29:08.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:29:09.156 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-04T17:29:09.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:29:12.220 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-03-04T17:29:16.420 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-03-04T17:29:17.201 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-04T17:29:17.201 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-97-generic"... 2024-03-04T17:29:17.201 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-03-04T17:29:17.249 INFO:teuthology.orchestra.run.smithi047.stdout:5.15.0-97-generic 2024-03-04T17:29:17.250 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-97-generic vs 5.15.0-97-generic 2024-03-04T17:29:17.250 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T17:29:17.250 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-04T17:29:18.254 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-04T17:29:18.255 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-04T17:29:18.346 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-03-04T17:29:18.392 DEBUG:teuthology.parallel:result is None 2024-03-04T17:29:27.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-04T17:29:27.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:29:27.631 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-03-04T17:29:28.197 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-04T17:29:28.197 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-97-generic"... 2024-03-04T17:29:28.198 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-03-04T17:29:28.246 INFO:teuthology.orchestra.run.smithi152.stdout:5.15.0-97-generic 2024-03-04T17:29:28.247 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-97-generic vs 5.15.0-97-generic 2024-03-04T17:29:28.247 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T17:29:28.247 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-04T17:29:29.248 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-04T17:29:29.248 DEBUG:teuthology.orchestra.run.smithi152:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-04T17:29:29.325 INFO:teuthology.orchestra.run.smithi152.stdout:ttyS1 2024-03-04T17:29:29.375 DEBUG:teuthology.parallel:result is None 2024-03-04T17:29:29.378 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-04T17:29:29.389 INFO:teuthology.task.internal:Creating test directory... 2024-03-04T17:29:29.390 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T17:29:29.405 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T17:29:29.412 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-04T17:29:29.454 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-04T17:29:29.493 INFO:teuthology.task.internal:Creating archive directory... 2024-03-04T17:29:29.493 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T17:29:29.511 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T17:29:29.541 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-04T17:29:29.569 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-04T17:29:29.569 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T17:29:29.573 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T17:29:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T17:29:29.594 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T17:29:29.602 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T17:29:29.602 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T17:29:29.606 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-04T17:29:29.613 INFO:teuthology.task.internal:Configuring sudo... 2024-03-04T17:29:29.613 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T17:29:29.647 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T17:29:29.669 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-04T17:29:29.680 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-04T17:29:29.680 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T17:29:29.707 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T17:29:29.724 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T17:29:29.758 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T17:29:29.807 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:29:29.807 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T17:29:29.876 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T17:29:29.883 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T17:29:29.930 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:29:29.930 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T17:29:30.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-03-04T17:29:30.035 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-03-04T17:29:30.105 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-04T17:29:30.125 INFO:teuthology.task.internal:Starting timer... 2024-03-04T17:29:30.125 INFO:teuthology.run_tasks:Running task pcp... 2024-03-04T17:29:30.143 INFO:teuthology.run_tasks:Running task selinux... 2024-03-04T17:29:30.174 DEBUG:teuthology.task.selinux:Excluding smithi047: OS 'ubuntu' does not support SELinux 2024-03-04T17:29:30.174 DEBUG:teuthology.task.selinux:Excluding smithi152: OS 'ubuntu' does not support SELinux 2024-03-04T17:29:30.175 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-04T17:29:30.184 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-04T17:29:30.184 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-04T17:29:30.184 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-04T17:29:30.209 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-04T17:29:30.305 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-04T17:29:30.306 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi152.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-04T17:37:10.830 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com')] 2024-03-04T17:37:10.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-04T17:37:10.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:37:10.915 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-03-04T17:37:10.992 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-04T17:37:10.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-04T17:37:10.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:37:11.070 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-03-04T17:37:11.148 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-04T17:37:11.148 INFO:teuthology.run_tasks:Running task clock... 2024-03-04T17:37:11.158 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-04T17:37:11.158 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T17:37:11.159 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T17:37:11.161 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T17:37:11.161 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T17:37:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-04T17:37:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: Command line: ntpd -gq 2024-03-04T17:37:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: ---------------------------------------------------- 2024-03-04T17:37:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: ntp-4 is maintained by Network Time Foundation, 2024-03-04T17:37:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-04T17:37:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: corporation. Support and training for ntp-4 are 2024-03-04T17:37:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: available at https://www.nwtime.org/support 2024-03-04T17:37:11.186 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: ---------------------------------------------------- 2024-03-04T17:37:11.187 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: proto: precision = 0.069 usec (-24) 2024-03-04T17:37:11.187 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: basedate set to 2022-02-04 2024-03-04T17:37:11.187 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: gps base set to 2022-02-06 (week 2196) 2024-03-04T17:37:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T17:37:11.188 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T17:37:11.188 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: restrict ::: KOD does nothing without LIMITED. 2024-03-04T17:37:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-04T17:37:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: Listen and drop on 0 v6wildcard [::]:123 2024-03-04T17:37:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-04T17:37:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: Listen normally on 2 lo 127.0.0.1:123 2024-03-04T17:37:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: Listen normally on 3 ens1f0 172.21.15.47:123 2024-03-04T17:37:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: Listen normally on 4 lo [::1]:123 2024-03-04T17:37:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:148e%4]:123 2024-03-04T17:37:11.189 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:11 ntpd[17076]: Listening on routing socket on fd #22 for interface updates 2024-03-04T17:37:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-04T17:37:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: Command line: ntpd -gq 2024-03-04T17:37:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: ---------------------------------------------------- 2024-03-04T17:37:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: ntp-4 is maintained by Network Time Foundation, 2024-03-04T17:37:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-04T17:37:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: corporation. Support and training for ntp-4 are 2024-03-04T17:37:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: available at https://www.nwtime.org/support 2024-03-04T17:37:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: ---------------------------------------------------- 2024-03-04T17:37:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: proto: precision = 0.043 usec (-24) 2024-03-04T17:37:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: basedate set to 2022-02-04 2024-03-04T17:37:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: gps base set to 2022-02-06 (week 2196) 2024-03-04T17:37:11.219 INFO:teuthology.orchestra.run.smithi152.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T17:37:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T17:37:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: restrict ::: KOD does nothing without LIMITED. 2024-03-04T17:37:11.219 INFO:teuthology.orchestra.run.smithi152.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-04T17:37:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: Listen and drop on 0 v6wildcard [::]:123 2024-03-04T17:37:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: Listen normally on 2 lo 127.0.0.1:123 2024-03-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: Listen normally on 3 enp3s0f1 172.21.15.152:123 2024-03-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: Listen normally on 4 lo [::1]:123 2024-03-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6de3%5]:123 2024-03-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:11 ntpd[17168]: Listening on routing socket on fd #22 for interface updates 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:12.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:12.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:12.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:12 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:14.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:14.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:14.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:14.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:14.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:14.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:14.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:14 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:14.218 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:14 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:14.218 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:14 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:14.218 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:14 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:14.218 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:14 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:14.218 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:14 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:14.218 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:14 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:15.217 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:15.217 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:15 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:16.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:16.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:16.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:16 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:16.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:16.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:16.221 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:16 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:18.188 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:18.188 INFO:teuthology.orchestra.run.smithi047.stdout: 4 Mar 17:37:18 ntpd[17076]: ntpd: time slew -0.001048 s 2024-03-04T17:37:18.188 INFO:teuthology.orchestra.run.smithi047.stdout:ntpd: time slew -0.001048s 2024-03-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi047.stderr: 4 Mar 17:37:18 ntpd[17076]: can't open /var/log/ntpstats/loopstats.20240304: Permission denied 2024-03-04T17:37:18.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:18.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:18 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:18.271 INFO:teuthology.orchestra.run.smithi047.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T17:37:18.271 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================== 2024-03-04T17:37:18.271 INFO:teuthology.orchestra.run.smithi047.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:37:18.271 INFO:teuthology.orchestra.run.smithi047.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:37:18.271 INFO:teuthology.orchestra.run.smithi047.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:37:18.271 INFO:teuthology.orchestra.run.smithi047.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:37:20.219 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:37:20.220 INFO:teuthology.orchestra.run.smithi152.stdout: 4 Mar 17:37:20 ntpd[17168]: ntpd: time slew -0.000626 s 2024-03-04T17:37:20.220 INFO:teuthology.orchestra.run.smithi152.stdout:ntpd: time slew -0.000626s 2024-03-04T17:37:20.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-04T17:37:20.220 INFO:teuthology.orchestra.run.smithi152.stderr: 4 Mar 17:37:20 ntpd[17168]: can't open /var/log/ntpstats/loopstats.20240304: Permission denied 2024-03-04T17:37:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T17:37:20.284 INFO:teuthology.orchestra.run.smithi152.stdout:============================================================================== 2024-03-04T17:37:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:37:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:37:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:37:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:37:20.286 INFO:teuthology.run_tasks:Running task install... 2024-03-04T17:37:20.294 DEBUG:teuthology.task.install:project ceph 2024-03-04T17:37:20.294 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}} 2024-03-04T17:37:20.295 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'} 2024-03-04T17:37:20.295 INFO:teuthology.task.install:Using flavor: default 2024-03-04T17:37:20.300 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-04T17:37:20.300 INFO:teuthology.task.install:extra packages: [] 2024-03-04T17:37:20.300 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-key list | grep Ceph 2024-03-04T17:37:20.301 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-key list | grep Ceph 2024-03-04T17:37:20.364 INFO:teuthology.orchestra.run.smithi047.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-04T17:37:20.386 INFO:teuthology.orchestra.run.smithi152.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-04T17:37:20.394 INFO:teuthology.orchestra.run.smithi047.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-04T17:37:20.394 INFO:teuthology.orchestra.run.smithi047.stdout:uid [ unknown] Ceph.com (release key) 2024-03-04T17:37:20.395 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-04T17:37:20.395 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-03-04T17:37:20.395 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:37:20.412 INFO:teuthology.orchestra.run.smithi152.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-04T17:37:20.413 INFO:teuthology.orchestra.run.smithi152.stdout:uid [ unknown] Ceph.com (release key) 2024-03-04T17:37:20.413 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-04T17:37:20.413 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-03-04T17:37:20.414 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:37:20.539 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default/ 2024-03-04T17:37:20.539 INFO:teuthology.task.install.deb:Package version is 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:37:20.551 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default/ 2024-03-04T17:37:20.551 INFO:teuthology.task.install.deb:Package version is 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:37:20.932 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:37:20.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-04T17:37:20.941 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:37:20.941 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-04T17:37:20.946 DEBUG:teuthology.orchestra.run.smithi047:> sudo apt-get update 2024-03-04T17:37:20.954 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-get update 2024-03-04T17:37:21.120 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T17:37:21.236 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:37:21.248 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:37:21.248 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T17:37:21.318 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T17:37:21.321 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T17:37:21.390 INFO:teuthology.orchestra.run.smithi047.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy InRelease 2024-03-04T17:37:21.399 INFO:teuthology.orchestra.run.smithi047.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T17:37:21.400 INFO:teuthology.orchestra.run.smithi152.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy InRelease 2024-03-04T17:37:21.402 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T17:37:21.468 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-04T17:37:21.477 INFO:teuthology.orchestra.run.smithi152.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-04T17:37:21.550 INFO:teuthology.orchestra.run.smithi047.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-04T17:37:21.557 INFO:teuthology.orchestra.run.smithi152.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-04T17:37:21.629 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-04T17:37:21.636 INFO:teuthology.orchestra.run.smithi152.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-04T17:37:21.714 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,395 B] 2024-03-04T17:37:21.796 INFO:teuthology.orchestra.run.smithi152.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,395 B] 2024-03-04T17:37:21.801 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 28.9 kB in 1s (43.7 kB/s) 2024-03-04T17:37:21.880 INFO:teuthology.orchestra.run.smithi152.stdout:Fetched 28.9 kB in 1s (39.3 kB/s) 2024-03-04T17:37:22.762 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-03-04T17:37:22.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1871-g3aec3903-1jammy cephadm=19.0.0-1871-g3aec3903-1jammy ceph-mds=19.0.0-1871-g3aec3903-1jammy ceph-mgr=19.0.0-1871-g3aec3903-1jammy ceph-common=19.0.0-1871-g3aec3903-1jammy ceph-fuse=19.0.0-1871-g3aec3903-1jammy ceph-test=19.0.0-1871-g3aec3903-1jammy ceph-volume=19.0.0-1871-g3aec3903-1jammy radosgw=19.0.0-1871-g3aec3903-1jammy python3-rados=19.0.0-1871-g3aec3903-1jammy python3-rgw=19.0.0-1871-g3aec3903-1jammy python3-cephfs=19.0.0-1871-g3aec3903-1jammy python3-rbd=19.0.0-1871-g3aec3903-1jammy libcephfs2=19.0.0-1871-g3aec3903-1jammy libcephfs-dev=19.0.0-1871-g3aec3903-1jammy librados2=19.0.0-1871-g3aec3903-1jammy librbd1=19.0.0-1871-g3aec3903-1jammy rbd-fuse=19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:37:22.839 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-03-04T17:37:22.851 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2024-03-04T17:37:22.866 DEBUG:teuthology.orchestra.run.smithi152:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1871-g3aec3903-1jammy cephadm=19.0.0-1871-g3aec3903-1jammy ceph-mds=19.0.0-1871-g3aec3903-1jammy ceph-mgr=19.0.0-1871-g3aec3903-1jammy ceph-common=19.0.0-1871-g3aec3903-1jammy ceph-fuse=19.0.0-1871-g3aec3903-1jammy ceph-test=19.0.0-1871-g3aec3903-1jammy ceph-volume=19.0.0-1871-g3aec3903-1jammy radosgw=19.0.0-1871-g3aec3903-1jammy python3-rados=19.0.0-1871-g3aec3903-1jammy python3-rgw=19.0.0-1871-g3aec3903-1jammy python3-cephfs=19.0.0-1871-g3aec3903-1jammy python3-rbd=19.0.0-1871-g3aec3903-1jammy libcephfs2=19.0.0-1871-g3aec3903-1jammy libcephfs-dev=19.0.0-1871-g3aec3903-1jammy librados2=19.0.0-1871-g3aec3903-1jammy librbd1=19.0.0-1871-g3aec3903-1jammy rbd-fuse=19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:37:22.917 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2024-03-04T17:37:23.028 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2024-03-04T17:37:23.029 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2024-03-04T17:37:23.087 INFO:teuthology.orchestra.run.smithi152.stdout:Building dependency tree... 2024-03-04T17:37:23.088 INFO:teuthology.orchestra.run.smithi152.stdout:Reading state information... 2024-03-04T17:37:23.216 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:37:23.216 INFO:teuthology.orchestra.run.smithi047.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:37:23.216 INFO:teuthology.orchestra.run.smithi047.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:37:23.216 INFO:teuthology.orchestra.run.smithi047.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:37:23.217 INFO:teuthology.orchestra.run.smithi047.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:37:23.217 INFO:teuthology.orchestra.run.smithi047.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:37:23.217 INFO:teuthology.orchestra.run.smithi047.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:37:23.218 INFO:teuthology.orchestra.run.smithi047.stdout:The following additional packages will be installed: 2024-03-04T17:37:23.218 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-04T17:37:23.218 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-04T17:37:23.219 INFO:teuthology.orchestra.run.smithi047.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T17:37:23.219 INFO:teuthology.orchestra.run.smithi047.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-04T17:37:23.219 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-04T17:37:23.219 INFO:teuthology.orchestra.run.smithi047.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-04T17:37:23.219 INFO:teuthology.orchestra.run.smithi047.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-04T17:37:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet zip 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout:Suggested packages: 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-04T17:37:23.221 INFO:teuthology.orchestra.run.smithi047.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-04T17:37:23.222 INFO:teuthology.orchestra.run.smithi047.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-04T17:37:23.222 INFO:teuthology.orchestra.run.smithi047.stdout:Recommended packages: 2024-03-04T17:37:23.222 INFO:teuthology.orchestra.run.smithi047.stdout: btrfs-tools 2024-03-04T17:37:23.245 INFO:teuthology.orchestra.run.smithi152.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:37:23.245 INFO:teuthology.orchestra.run.smithi152.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:37:23.245 INFO:teuthology.orchestra.run.smithi152.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:37:23.245 INFO:teuthology.orchestra.run.smithi152.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:37:23.245 INFO:teuthology.orchestra.run.smithi152.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:37:23.246 INFO:teuthology.orchestra.run.smithi152.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:37:23.246 INFO:teuthology.orchestra.run.smithi152.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:37:23.247 INFO:teuthology.orchestra.run.smithi152.stdout:The following additional packages will be installed: 2024-03-04T17:37:23.247 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-04T17:37:23.247 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T17:37:23.248 INFO:teuthology.orchestra.run.smithi152.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T17:37:23.249 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-04T17:37:23.249 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-04T17:37:23.249 INFO:teuthology.orchestra.run.smithi152.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-04T17:37:23.249 INFO:teuthology.orchestra.run.smithi152.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-04T17:37:23.249 INFO:teuthology.orchestra.run.smithi152.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-04T17:37:23.249 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-04T17:37:23.249 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-04T17:37:23.249 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet zip 2024-03-04T17:37:23.249 INFO:teuthology.orchestra.run.smithi152.stdout:Suggested packages: 2024-03-04T17:37:23.250 INFO:teuthology.orchestra.run.smithi152.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-04T17:37:23.250 INFO:teuthology.orchestra.run.smithi152.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-04T17:37:23.250 INFO:teuthology.orchestra.run.smithi152.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-04T17:37:23.250 INFO:teuthology.orchestra.run.smithi152.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-04T17:37:23.250 INFO:teuthology.orchestra.run.smithi152.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-04T17:37:23.250 INFO:teuthology.orchestra.run.smithi152.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-04T17:37:23.250 INFO:teuthology.orchestra.run.smithi152.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-04T17:37:23.250 INFO:teuthology.orchestra.run.smithi152.stdout:Recommended packages: 2024-03-04T17:37:23.250 INFO:teuthology.orchestra.run.smithi152.stdout: btrfs-tools 2024-03-04T17:37:23.328 INFO:teuthology.orchestra.run.smithi047.stdout:The following NEW packages will be installed: 2024-03-04T17:37:23.328 INFO:teuthology.orchestra.run.smithi047.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-04T17:37:23.328 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-04T17:37:23.329 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-04T17:37:23.329 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-04T17:37:23.329 INFO:teuthology.orchestra.run.smithi047.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-04T17:37:23.329 INFO:teuthology.orchestra.run.smithi047.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-04T17:37:23.330 INFO:teuthology.orchestra.run.smithi047.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-04T17:37:23.330 INFO:teuthology.orchestra.run.smithi047.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-04T17:37:23.330 INFO:teuthology.orchestra.run.smithi047.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-04T17:37:23.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-04T17:37:23.331 INFO:teuthology.orchestra.run.smithi047.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-04T17:37:23.350 INFO:teuthology.orchestra.run.smithi152.stdout:The following NEW packages will be installed: 2024-03-04T17:37:23.351 INFO:teuthology.orchestra.run.smithi152.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-04T17:37:23.351 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-04T17:37:23.352 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T17:37:23.353 INFO:teuthology.orchestra.run.smithi152.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-04T17:37:23.354 INFO:teuthology.orchestra.run.smithi152.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-04T17:37:23.396 INFO:teuthology.orchestra.run.smithi047.stdout:0 upgraded, 118 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T17:37:23.396 INFO:teuthology.orchestra.run.smithi047.stdout:Need to get 177 MB of archives. 2024-03-04T17:37:23.397 INFO:teuthology.orchestra.run.smithi047.stdout:After this operation, 734 MB of additional disk space will be used. 2024-03-04T17:37:23.397 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-04T17:37:23.416 INFO:teuthology.orchestra.run.smithi152.stdout:0 upgraded, 118 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T17:37:23.416 INFO:teuthology.orchestra.run.smithi152.stdout:Need to get 177 MB of archives. 2024-03-04T17:37:23.416 INFO:teuthology.orchestra.run.smithi152.stdout:After this operation, 734 MB of additional disk space will be used. 2024-03-04T17:37:23.416 INFO:teuthology.orchestra.run.smithi152.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-04T17:37:23.499 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-04T17:37:23.518 INFO:teuthology.orchestra.run.smithi152.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-04T17:37:23.521 INFO:teuthology.orchestra.run.smithi152.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-04T17:37:23.540 INFO:teuthology.orchestra.run.smithi152.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-04T17:37:23.547 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-04T17:37:23.552 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-04T17:37:23.606 INFO:teuthology.orchestra.run.smithi152.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-04T17:37:23.611 INFO:teuthology.orchestra.run.smithi152.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-04T17:37:23.625 INFO:teuthology.orchestra.run.smithi152.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-04T17:37:23.629 INFO:teuthology.orchestra.run.smithi152.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-04T17:37:23.629 INFO:teuthology.orchestra.run.smithi152.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-04T17:37:23.630 INFO:teuthology.orchestra.run.smithi152.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-04T17:37:23.634 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-04T17:37:23.639 INFO:teuthology.orchestra.run.smithi152.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-04T17:37:23.641 INFO:teuthology.orchestra.run.smithi152.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-04T17:37:23.642 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-04T17:37:23.643 INFO:teuthology.orchestra.run.smithi152.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-04T17:37:23.647 INFO:teuthology.orchestra.run.smithi152.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-04T17:37:23.660 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-04T17:37:23.666 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-04T17:37:23.666 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-04T17:37:23.668 INFO:teuthology.orchestra.run.smithi047.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-04T17:37:23.673 INFO:teuthology.orchestra.run.smithi152.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-04T17:37:23.676 INFO:teuthology.orchestra.run.smithi152.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-04T17:37:23.678 INFO:teuthology.orchestra.run.smithi047.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-04T17:37:23.679 INFO:teuthology.orchestra.run.smithi152.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-04T17:37:23.681 INFO:teuthology.orchestra.run.smithi152.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-04T17:37:23.681 INFO:teuthology.orchestra.run.smithi047.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-04T17:37:23.682 INFO:teuthology.orchestra.run.smithi152.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-04T17:37:23.682 INFO:teuthology.orchestra.run.smithi152.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-04T17:37:23.683 INFO:teuthology.orchestra.run.smithi152.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-04T17:37:23.684 INFO:teuthology.orchestra.run.smithi047.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-04T17:37:23.684 INFO:teuthology.orchestra.run.smithi152.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-04T17:37:23.685 INFO:teuthology.orchestra.run.smithi152.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-04T17:37:23.685 INFO:teuthology.orchestra.run.smithi047.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-04T17:37:23.689 INFO:teuthology.orchestra.run.smithi047.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-04T17:37:23.691 INFO:teuthology.orchestra.run.smithi047.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-04T17:37:23.694 INFO:teuthology.orchestra.run.smithi047.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-04T17:37:23.698 INFO:teuthology.orchestra.run.smithi152.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-04T17:37:23.705 INFO:teuthology.orchestra.run.smithi152.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-04T17:37:23.705 INFO:teuthology.orchestra.run.smithi152.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-04T17:37:23.706 INFO:teuthology.orchestra.run.smithi152.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-04T17:37:23.709 INFO:teuthology.orchestra.run.smithi047.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-04T17:37:23.709 INFO:teuthology.orchestra.run.smithi152.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-04T17:37:23.710 INFO:teuthology.orchestra.run.smithi047.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-04T17:37:23.710 INFO:teuthology.orchestra.run.smithi152.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-04T17:37:23.711 INFO:teuthology.orchestra.run.smithi047.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-04T17:37:23.711 INFO:teuthology.orchestra.run.smithi152.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-04T17:37:23.711 INFO:teuthology.orchestra.run.smithi152.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-04T17:37:23.712 INFO:teuthology.orchestra.run.smithi047.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-04T17:37:23.712 INFO:teuthology.orchestra.run.smithi047.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-04T17:37:23.713 INFO:teuthology.orchestra.run.smithi152.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-04T17:37:23.713 INFO:teuthology.orchestra.run.smithi047.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-04T17:37:23.713 INFO:teuthology.orchestra.run.smithi047.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-04T17:37:23.717 INFO:teuthology.orchestra.run.smithi152.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-04T17:37:23.729 INFO:teuthology.orchestra.run.smithi047.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-04T17:37:23.729 INFO:teuthology.orchestra.run.smithi047.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-04T17:37:23.730 INFO:teuthology.orchestra.run.smithi047.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-04T17:37:23.734 INFO:teuthology.orchestra.run.smithi152.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-04T17:37:23.736 INFO:teuthology.orchestra.run.smithi152.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-04T17:37:23.736 INFO:teuthology.orchestra.run.smithi152.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-04T17:37:23.737 INFO:teuthology.orchestra.run.smithi152.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-04T17:37:23.744 INFO:teuthology.orchestra.run.smithi152.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-04T17:37:23.744 INFO:teuthology.orchestra.run.smithi152.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-04T17:37:23.745 INFO:teuthology.orchestra.run.smithi152.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-04T17:37:23.746 INFO:teuthology.orchestra.run.smithi152.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-04T17:37:23.748 INFO:teuthology.orchestra.run.smithi152.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-04T17:37:23.751 INFO:teuthology.orchestra.run.smithi047.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-04T17:37:23.752 INFO:teuthology.orchestra.run.smithi047.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-04T17:37:23.752 INFO:teuthology.orchestra.run.smithi047.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-04T17:37:23.753 INFO:teuthology.orchestra.run.smithi047.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-04T17:37:23.754 INFO:teuthology.orchestra.run.smithi047.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-04T17:37:23.755 INFO:teuthology.orchestra.run.smithi047.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-04T17:37:23.755 INFO:teuthology.orchestra.run.smithi047.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-04T17:37:23.759 INFO:teuthology.orchestra.run.smithi047.stdout:Get:35 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1871-g3aec3903-1jammy [3,547 kB] 2024-03-04T17:37:23.762 INFO:teuthology.orchestra.run.smithi152.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-04T17:37:23.763 INFO:teuthology.orchestra.run.smithi152.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-04T17:37:23.765 INFO:teuthology.orchestra.run.smithi152.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-04T17:37:23.766 INFO:teuthology.orchestra.run.smithi152.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-04T17:37:23.771 INFO:teuthology.orchestra.run.smithi047.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-04T17:37:23.772 INFO:teuthology.orchestra.run.smithi047.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-04T17:37:23.772 INFO:teuthology.orchestra.run.smithi047.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-04T17:37:23.777 INFO:teuthology.orchestra.run.smithi152.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1871-g3aec3903-1jammy [3,547 kB] 2024-03-04T17:37:23.782 INFO:teuthology.orchestra.run.smithi152.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-04T17:37:23.783 INFO:teuthology.orchestra.run.smithi152.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-04T17:37:23.784 INFO:teuthology.orchestra.run.smithi152.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-04T17:37:23.787 INFO:teuthology.orchestra.run.smithi152.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-04T17:37:23.801 INFO:teuthology.orchestra.run.smithi047.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-04T17:37:23.801 INFO:teuthology.orchestra.run.smithi047.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-04T17:37:23.802 INFO:teuthology.orchestra.run.smithi047.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-04T17:37:23.803 INFO:teuthology.orchestra.run.smithi047.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-04T17:37:23.805 INFO:teuthology.orchestra.run.smithi047.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-04T17:37:23.809 INFO:teuthology.orchestra.run.smithi047.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-04T17:37:23.810 INFO:teuthology.orchestra.run.smithi047.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-04T17:37:23.841 INFO:teuthology.orchestra.run.smithi047.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-04T17:37:23.842 INFO:teuthology.orchestra.run.smithi047.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-04T17:37:23.847 INFO:teuthology.orchestra.run.smithi047.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-04T17:37:23.853 INFO:teuthology.orchestra.run.smithi152.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-04T17:37:23.854 INFO:teuthology.orchestra.run.smithi152.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-04T17:37:23.855 INFO:teuthology.orchestra.run.smithi152.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-04T17:37:23.856 INFO:teuthology.orchestra.run.smithi152.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-04T17:37:23.857 INFO:teuthology.orchestra.run.smithi152.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-04T17:37:23.884 INFO:teuthology.orchestra.run.smithi047.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-04T17:37:23.885 INFO:teuthology.orchestra.run.smithi047.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-04T17:37:23.891 INFO:teuthology.orchestra.run.smithi047.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-04T17:37:23.891 INFO:teuthology.orchestra.run.smithi152.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-04T17:37:23.896 INFO:teuthology.orchestra.run.smithi152.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-04T17:37:23.897 INFO:teuthology.orchestra.run.smithi152.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-04T17:37:23.984 INFO:teuthology.orchestra.run.smithi152.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-04T17:37:23.985 INFO:teuthology.orchestra.run.smithi152.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-04T17:37:23.986 INFO:teuthology.orchestra.run.smithi152.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-04T17:37:23.986 INFO:teuthology.orchestra.run.smithi152.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-04T17:37:23.987 INFO:teuthology.orchestra.run.smithi152.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-04T17:37:23.987 INFO:teuthology.orchestra.run.smithi152.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-04T17:37:23.988 INFO:teuthology.orchestra.run.smithi152.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-04T17:37:23.989 INFO:teuthology.orchestra.run.smithi152.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-04T17:37:23.994 INFO:teuthology.orchestra.run.smithi152.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-04T17:37:24.006 INFO:teuthology.orchestra.run.smithi047.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-04T17:37:24.007 INFO:teuthology.orchestra.run.smithi047.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-04T17:37:24.010 INFO:teuthology.orchestra.run.smithi047.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-04T17:37:24.010 INFO:teuthology.orchestra.run.smithi047.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-04T17:37:24.012 INFO:teuthology.orchestra.run.smithi047.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-04T17:37:24.013 INFO:teuthology.orchestra.run.smithi152.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-04T17:37:24.016 INFO:teuthology.orchestra.run.smithi152.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-04T17:37:24.037 INFO:teuthology.orchestra.run.smithi047.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-04T17:37:24.039 INFO:teuthology.orchestra.run.smithi047.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-04T17:37:24.040 INFO:teuthology.orchestra.run.smithi047.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-04T17:37:24.040 INFO:teuthology.orchestra.run.smithi152.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-04T17:37:24.046 INFO:teuthology.orchestra.run.smithi152.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-04T17:37:24.051 INFO:teuthology.orchestra.run.smithi152.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-04T17:37:24.053 INFO:teuthology.orchestra.run.smithi152.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-04T17:37:24.054 INFO:teuthology.orchestra.run.smithi152.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-04T17:37:24.057 INFO:teuthology.orchestra.run.smithi047.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-04T17:37:24.059 INFO:teuthology.orchestra.run.smithi047.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-04T17:37:24.063 INFO:teuthology.orchestra.run.smithi152.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-04T17:37:24.064 INFO:teuthology.orchestra.run.smithi152.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-04T17:37:24.067 INFO:teuthology.orchestra.run.smithi047.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-04T17:37:24.070 INFO:teuthology.orchestra.run.smithi152.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-04T17:37:24.070 INFO:teuthology.orchestra.run.smithi152.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-04T17:37:24.071 INFO:teuthology.orchestra.run.smithi047.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-04T17:37:24.072 INFO:teuthology.orchestra.run.smithi152.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-04T17:37:24.072 INFO:teuthology.orchestra.run.smithi047.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-04T17:37:24.074 INFO:teuthology.orchestra.run.smithi047.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-04T17:37:24.086 INFO:teuthology.orchestra.run.smithi047.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-04T17:37:24.087 INFO:teuthology.orchestra.run.smithi047.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-04T17:37:24.091 INFO:teuthology.orchestra.run.smithi152.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-04T17:37:24.092 INFO:teuthology.orchestra.run.smithi047.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-04T17:37:24.094 INFO:teuthology.orchestra.run.smithi152.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-04T17:37:24.096 INFO:teuthology.orchestra.run.smithi152.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-04T17:37:24.101 INFO:teuthology.orchestra.run.smithi047.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-04T17:37:24.103 INFO:teuthology.orchestra.run.smithi047.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-04T17:37:24.106 INFO:teuthology.orchestra.run.smithi047.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-04T17:37:24.108 INFO:teuthology.orchestra.run.smithi152.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-04T17:37:24.108 INFO:teuthology.orchestra.run.smithi152.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-04T17:37:24.109 INFO:teuthology.orchestra.run.smithi152.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-04T17:37:24.112 INFO:teuthology.orchestra.run.smithi047.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-04T17:37:24.117 INFO:teuthology.orchestra.run.smithi047.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-04T17:37:24.119 INFO:teuthology.orchestra.run.smithi047.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-04T17:37:24.120 INFO:teuthology.orchestra.run.smithi152.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-04T17:37:24.122 INFO:teuthology.orchestra.run.smithi152.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-04T17:37:24.122 INFO:teuthology.orchestra.run.smithi047.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-04T17:37:24.160 INFO:teuthology.orchestra.run.smithi047.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-04T17:37:24.163 INFO:teuthology.orchestra.run.smithi047.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-04T17:37:24.170 INFO:teuthology.orchestra.run.smithi047.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-04T17:37:24.175 INFO:teuthology.orchestra.run.smithi047.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-04T17:37:24.177 INFO:teuthology.orchestra.run.smithi047.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-04T17:37:24.181 INFO:teuthology.orchestra.run.smithi047.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-04T17:37:24.183 INFO:teuthology.orchestra.run.smithi047.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-04T17:37:24.185 INFO:teuthology.orchestra.run.smithi047.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-04T17:37:24.197 INFO:teuthology.orchestra.run.smithi047.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-04T17:37:24.197 INFO:teuthology.orchestra.run.smithi047.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-04T17:37:24.198 INFO:teuthology.orchestra.run.smithi047.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-04T17:37:24.207 INFO:teuthology.orchestra.run.smithi047.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-04T17:37:24.209 INFO:teuthology.orchestra.run.smithi047.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-04T17:37:25.465 INFO:teuthology.orchestra.run.smithi047.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1871-g3aec3903-1jammy [3,217 kB] 2024-03-04T17:37:26.474 INFO:teuthology.orchestra.run.smithi047.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1871-g3aec3903-1jammy [710 kB] 2024-03-04T17:37:26.693 INFO:teuthology.orchestra.run.smithi047.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1871-g3aec3903-1jammy [356 kB] 2024-03-04T17:37:26.769 INFO:teuthology.orchestra.run.smithi047.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1871-g3aec3903-1jammy [32.8 kB] 2024-03-04T17:37:26.771 INFO:teuthology.orchestra.run.smithi047.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1871-g3aec3903-1jammy [180 kB] 2024-03-04T17:37:26.840 INFO:teuthology.orchestra.run.smithi047.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1871-g3aec3903-1jammy [64.4 kB] 2024-03-04T17:37:26.842 INFO:teuthology.orchestra.run.smithi047.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1871-g3aec3903-1jammy [330 kB] 2024-03-04T17:37:26.978 INFO:teuthology.orchestra.run.smithi047.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1871-g3aec3903-1jammy [6,634 kB] 2024-03-04T17:37:28.840 INFO:teuthology.orchestra.run.smithi152.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1871-g3aec3903-1jammy [3,217 kB] 2024-03-04T17:37:28.890 INFO:teuthology.orchestra.run.smithi047.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1871-g3aec3903-1jammy [112 kB] 2024-03-04T17:37:28.892 INFO:teuthology.orchestra.run.smithi047.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1871-g3aec3903-1jammy [466 kB] 2024-03-04T17:37:29.001 INFO:teuthology.orchestra.run.smithi047.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1871-g3aec3903-1jammy [24.2 MB] 2024-03-04T17:37:33.333 INFO:teuthology.orchestra.run.smithi152.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1871-g3aec3903-1jammy [710 kB] 2024-03-04T17:37:34.136 INFO:teuthology.orchestra.run.smithi152.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1871-g3aec3903-1jammy [356 kB] 2024-03-04T17:37:34.516 INFO:teuthology.orchestra.run.smithi152.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1871-g3aec3903-1jammy [32.8 kB] 2024-03-04T17:37:34.546 INFO:teuthology.orchestra.run.smithi152.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1871-g3aec3903-1jammy [180 kB] 2024-03-04T17:37:34.787 INFO:teuthology.orchestra.run.smithi152.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1871-g3aec3903-1jammy [64.4 kB] 2024-03-04T17:37:34.901 INFO:teuthology.orchestra.run.smithi152.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1871-g3aec3903-1jammy [330 kB] 2024-03-04T17:37:35.336 INFO:teuthology.orchestra.run.smithi152.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1871-g3aec3903-1jammy [6,634 kB] 2024-03-04T17:37:41.017 INFO:teuthology.orchestra.run.smithi047.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1871-g3aec3903-1jammy [5,077 kB] 2024-03-04T17:37:42.423 INFO:teuthology.orchestra.run.smithi152.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1871-g3aec3903-1jammy [112 kB] 2024-03-04T17:37:42.526 INFO:teuthology.orchestra.run.smithi152.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1871-g3aec3903-1jammy [466 kB] 2024-03-04T17:37:42.945 INFO:teuthology.orchestra.run.smithi152.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1871-g3aec3903-1jammy [24.2 MB] 2024-03-04T17:37:43.905 INFO:teuthology.orchestra.run.smithi047.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1871-g3aec3903-1jammy [243 kB] 2024-03-04T17:37:44.055 INFO:teuthology.orchestra.run.smithi047.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1871-g3aec3903-1jammy [124 kB] 2024-03-04T17:37:44.096 INFO:teuthology.orchestra.run.smithi047.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1871-g3aec3903-1jammy [1,521 kB] 2024-03-04T17:37:44.923 INFO:teuthology.orchestra.run.smithi047.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1871-g3aec3903-1jammy [6,237 kB] 2024-03-04T17:37:47.995 INFO:teuthology.orchestra.run.smithi047.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1871-g3aec3903-1jammy [22.7 MB] 2024-03-04T17:37:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1871-g3aec3903-1jammy [5,077 kB] 2024-03-04T17:38:02.262 INFO:teuthology.orchestra.run.smithi152.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1871-g3aec3903-1jammy [243 kB] 2024-03-04T17:38:02.434 INFO:teuthology.orchestra.run.smithi152.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1871-g3aec3903-1jammy [124 kB] 2024-03-04T17:38:02.518 INFO:teuthology.orchestra.run.smithi152.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1871-g3aec3903-1jammy [1,521 kB] 2024-03-04T17:38:03.677 INFO:teuthology.orchestra.run.smithi152.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1871-g3aec3903-1jammy [6,237 kB] 2024-03-04T17:38:03.749 INFO:teuthology.orchestra.run.smithi047.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1871-g3aec3903-1jammy [14.0 kB] 2024-03-04T17:38:03.750 INFO:teuthology.orchestra.run.smithi047.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1871-g3aec3903-1jammy [866 kB] 2024-03-04T17:38:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1871-g3aec3903-1jammy [2,190 kB] 2024-03-04T17:38:05.280 INFO:teuthology.orchestra.run.smithi047.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1871-g3aec3903-1jammy [777 kB] 2024-03-04T17:38:05.745 INFO:teuthology.orchestra.run.smithi047.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1871-g3aec3903-1jammy [144 kB] 2024-03-04T17:38:05.819 INFO:teuthology.orchestra.run.smithi047.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1871-g3aec3903-1jammy [1,922 kB] 2024-03-04T17:38:06.819 INFO:teuthology.orchestra.run.smithi047.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1871-g3aec3903-1jammy [8,625 kB] 2024-03-04T17:38:08.439 INFO:teuthology.orchestra.run.smithi152.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1871-g3aec3903-1jammy [22.7 MB] 2024-03-04T17:38:11.758 INFO:teuthology.orchestra.run.smithi047.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1871-g3aec3903-1jammy [14.2 kB] 2024-03-04T17:38:11.760 INFO:teuthology.orchestra.run.smithi047.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1871-g3aec3903-1jammy [50.7 MB] 2024-03-04T17:38:23.673 INFO:teuthology.orchestra.run.smithi152.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1871-g3aec3903-1jammy [14.0 kB] 2024-03-04T17:38:23.674 INFO:teuthology.orchestra.run.smithi152.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1871-g3aec3903-1jammy [866 kB] 2024-03-04T17:38:23.987 INFO:teuthology.orchestra.run.smithi152.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1871-g3aec3903-1jammy [2,190 kB] 2024-03-04T17:38:24.704 INFO:teuthology.orchestra.run.smithi152.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1871-g3aec3903-1jammy [777 kB] 2024-03-04T17:38:24.969 INFO:teuthology.orchestra.run.smithi152.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1871-g3aec3903-1jammy [144 kB] 2024-03-04T17:38:25.007 INFO:teuthology.orchestra.run.smithi152.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1871-g3aec3903-1jammy [1,922 kB] 2024-03-04T17:38:25.691 INFO:teuthology.orchestra.run.smithi152.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1871-g3aec3903-1jammy [8,625 kB] 2024-03-04T17:38:28.960 INFO:teuthology.orchestra.run.smithi152.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1871-g3aec3903-1jammy [14.2 kB] 2024-03-04T17:38:28.960 INFO:teuthology.orchestra.run.smithi152.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1871-g3aec3903-1jammy [50.7 MB] 2024-03-04T17:38:37.948 INFO:teuthology.orchestra.run.smithi047.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1871-g3aec3903-1jammy [121 kB] 2024-03-04T17:38:37.951 INFO:teuthology.orchestra.run.smithi047.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1871-g3aec3903-1jammy [38.4 kB] 2024-03-04T17:38:37.952 INFO:teuthology.orchestra.run.smithi047.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1871-g3aec3903-1jammy [13.1 MB] 2024-03-04T17:38:46.226 INFO:teuthology.orchestra.run.smithi047.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1871-g3aec3903-1jammy [91.7 kB] 2024-03-04T17:38:46.830 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 177 MB in 1min 23s (2,135 kB/s) 2024-03-04T17:38:46.962 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-04T17:38:47.063 INFO:teuthology.orchestra.run.smithi047.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165994 files and directories currently installed.) 2024-03-04T17:38:47.069 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-04T17:38:47.153 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T17:38:47.506 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-04T17:38:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-04T17:38:47.572 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T17:38:47.867 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-04T17:38:47.894 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-04T17:38:47.934 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T17:38:48.496 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-04T17:38:48.523 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:38:48.596 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:38:48.675 INFO:teuthology.orchestra.run.smithi152.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1871-g3aec3903-1jammy [121 kB] 2024-03-04T17:38:48.678 INFO:teuthology.orchestra.run.smithi152.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1871-g3aec3903-1jammy [38.4 kB] 2024-03-04T17:38:48.679 INFO:teuthology.orchestra.run.smithi152.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1871-g3aec3903-1jammy [13.1 MB] 2024-03-04T17:38:49.015 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-04T17:38:49.042 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:38:49.082 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:38:49.393 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-04T17:38:49.420 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:38:49.451 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:38:49.813 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-04T17:38:49.840 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-04T17:38:49.879 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T17:38:50.199 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librados2. 2024-03-04T17:38:50.225 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../007-librados2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:50.265 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librados2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:50.676 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librbd1. 2024-03-04T17:38:50.692 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../008-librbd1_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:50.726 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librbd1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:51.146 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcephfs2. 2024-03-04T17:38:51.172 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:51.212 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcephfs2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:51.498 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rados. 2024-03-04T17:38:51.514 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../010-python3-rados_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:51.556 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rados (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:52.008 INFO:teuthology.orchestra.run.smithi152.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1871-g3aec3903-1jammy [91.7 kB] 2024-03-04T17:38:52.051 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-04T17:38:52.077 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:52.117 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-ceph-argparse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:52.319 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cephfs. 2024-03-04T17:38:52.335 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:52.386 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cephfs (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:52.587 INFO:teuthology.orchestra.run.smithi152.stdout:Fetched 177 MB in 1min 29s (1,996 kB/s) 2024-03-04T17:38:52.697 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-04T17:38:52.724 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:52.745 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-04T17:38:52.764 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:52.842 INFO:teuthology.orchestra.run.smithi152.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165994 files and directories currently installed.) 2024-03-04T17:38:52.847 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-04T17:38:52.943 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T17:38:53.083 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-04T17:38:53.110 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-04T17:38:53.150 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T17:38:53.289 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-04T17:38:53.316 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-04T17:38:53.355 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T17:38:53.469 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-prettytable. 2024-03-04T17:38:53.496 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-04T17:38:53.536 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-04T17:38:53.643 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-04T17:38:53.669 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-04T17:38:53.701 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T17:38:53.763 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rbd. 2024-03-04T17:38:53.779 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:53.813 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rbd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:54.141 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-04T17:38:54.163 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-04T17:38:54.168 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-04T17:38:54.189 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:38:54.208 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T17:38:54.238 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:38:54.485 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-04T17:38:54.512 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-04T17:38:54.552 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T17:38:54.625 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-04T17:38:54.651 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:38:54.691 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:38:54.863 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-04T17:38:54.890 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-04T17:38:54.930 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T17:38:55.020 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-04T17:38:55.046 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:38:55.086 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:38:55.250 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package lua5.1. 2024-03-04T17:38:55.265 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-04T17:38:55.299 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-04T17:38:55.432 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-04T17:38:55.458 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-04T17:38:55.498 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T17:38:55.619 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package lua-any. 2024-03-04T17:38:55.646 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-04T17:38:55.686 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-04T17:38:55.827 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package librados2. 2024-03-04T17:38:55.853 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../007-librados2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:55.893 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking librados2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:55.938 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package zip. 2024-03-04T17:38:55.965 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-04T17:38:56.030 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking zip (3.0-12build2) ... 2024-03-04T17:38:56.358 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package unzip. 2024-03-04T17:38:56.385 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-04T17:38:56.424 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-04T17:38:56.513 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package librbd1. 2024-03-04T17:38:56.539 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../008-librbd1_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:56.579 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking librbd1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:56.778 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package luarocks. 2024-03-04T17:38:56.805 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-04T17:38:56.836 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-04T17:38:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libcephfs2. 2024-03-04T17:38:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libcephfs2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:57.227 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package librgw2. 2024-03-04T17:38:57.254 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../025-librgw2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:57.294 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking librgw2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-rados. 2024-03-04T17:38:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../010-python3-rados_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-rados (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-04T17:38:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-ceph-argparse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rgw. 2024-03-04T17:38:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:57.913 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-cephfs. 2024-03-04T17:38:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-cephfs (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:58.249 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-04T17:38:58.276 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-04T17:38:58.315 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T17:38:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-04T17:38:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:58.585 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libradosstriper1. 2024-03-04T17:38:58.601 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:58.635 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libradosstriper1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:58.798 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-04T17:38:58.824 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-04T17:38:58.864 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T17:38:59.032 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-common. 2024-03-04T17:38:59.056 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../029-ceph-common_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:59.098 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:59.209 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-prettytable. 2024-03-04T17:38:59.236 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-04T17:38:59.276 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-04T17:38:59.520 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-rbd. 2024-03-04T17:38:59.535 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-rbd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:59.916 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-04T17:38:59.942 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-04T17:38:59.982 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T17:39:00.260 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-04T17:39:00.286 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-04T17:39:00.326 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T17:39:00.487 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-base. 2024-03-04T17:39:00.514 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../030-ceph-base_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:00.554 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-base (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:00.613 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-04T17:39:00.628 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-04T17:39:00.663 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T17:39:00.967 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package lua5.1. 2024-03-04T17:39:00.993 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-04T17:39:01.033 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-04T17:39:01.157 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-04T17:39:01.184 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-04T17:39:01.224 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-04T17:39:01.345 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package lua-any. 2024-03-04T17:39:01.372 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-04T17:39:01.412 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-04T17:39:01.527 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cheroot. 2024-03-04T17:39:01.553 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-04T17:39:01.593 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T17:39:01.648 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package zip. 2024-03-04T17:39:01.663 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-04T17:39:01.698 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking zip (3.0-12build2) ... 2024-03-04T17:39:01.896 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-04T17:39:01.923 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-04T17:39:01.954 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-04T17:39:02.026 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package unzip. 2024-03-04T17:39:02.053 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-04T17:39:02.093 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-04T17:39:02.232 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-04T17:39:02.259 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-04T17:39:02.299 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-04T17:39:02.489 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package luarocks. 2024-03-04T17:39:02.515 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-04T17:39:02.555 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-04T17:39:02.585 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-04T17:39:02.601 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-04T17:39:02.643 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-04T17:39:02.946 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-tempora. 2024-03-04T17:39:02.947 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package librgw2. 2024-03-04T17:39:02.972 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-04T17:39:02.974 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../025-librgw2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:03.012 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-04T17:39:03.014 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking librgw2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:03.274 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-portend. 2024-03-04T17:39:03.300 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-04T17:39:03.332 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-04T17:39:03.559 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-rgw. 2024-03-04T17:39:03.574 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:03.608 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-rgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:03.609 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-04T17:39:03.636 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-04T17:39:03.676 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-04T17:39:03.953 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-04T17:39:03.980 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-04T17:39:03.995 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-04T17:39:04.020 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T17:39:04.022 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-04T17:39:04.063 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-04T17:39:04.331 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libradosstriper1. 2024-03-04T17:39:04.358 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:04.398 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libradosstriper1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:04.399 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-natsort. 2024-03-04T17:39:04.426 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-04T17:39:04.465 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-04T17:39:04.768 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-common. 2024-03-04T17:39:04.777 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-logutils. 2024-03-04T17:39:04.795 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../029-ceph-common_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:04.804 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-04T17:39:04.826 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:04.843 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-04T17:39:05.146 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-04T17:39:05.173 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-04T17:39:05.212 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-04T17:39:05.515 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-mako. 2024-03-04T17:39:05.542 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-04T17:39:05.573 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T17:39:05.876 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-04T17:39:05.904 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-04T17:39:05.934 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-04T17:39:06.132 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-base. 2024-03-04T17:39:06.159 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../030-ceph-base_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:06.217 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-base (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:06.237 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-04T17:39:06.264 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-04T17:39:06.304 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-04T17:39:06.615 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-webob. 2024-03-04T17:39:06.642 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-04T17:39:06.681 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-04T17:39:06.954 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-04T17:39:06.981 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-04T17:39:07.021 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-04T17:39:07.026 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-waitress. 2024-03-04T17:39:07.053 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-04T17:39:07.096 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T17:39:07.391 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-cheroot. 2024-03-04T17:39:07.412 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-tempita. 2024-03-04T17:39:07.417 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-04T17:39:07.439 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-04T17:39:07.466 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T17:39:07.478 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T17:39:07.788 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-paste. 2024-03-04T17:39:07.815 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-04T17:39:07.844 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-04T17:39:07.856 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T17:39:07.871 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-04T17:39:07.910 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-04T17:39:08.193 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-04T17:39:08.209 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-04T17:39:08.214 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-04T17:39:08.240 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-04T17:39:08.251 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T17:39:08.280 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-04T17:39:08.537 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-04T17:39:08.564 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-04T17:39:08.603 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-04T17:39:08.634 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-04T17:39:08.660 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-04T17:39:08.709 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-04T17:39:08.915 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-webtest. 2024-03-04T17:39:08.942 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-04T17:39:08.989 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-04T17:39:09.062 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-tempora. 2024-03-04T17:39:09.088 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-04T17:39:09.136 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-04T17:39:09.259 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pecan. 2024-03-04T17:39:09.275 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-04T17:39:09.309 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T17:39:09.440 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-portend. 2024-03-04T17:39:09.466 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-04T17:39:09.506 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-04T17:39:09.605 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-certifi. 2024-03-04T17:39:09.618 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-04T17:39:09.646 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-04T17:39:09.876 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-04T17:39:09.903 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-04T17:39:09.951 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-04T17:39:09.967 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-idna. 2024-03-04T17:39:09.993 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-04T17:39:10.033 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-04T17:39:10.305 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-04T17:39:10.331 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-04T17:39:10.344 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-urllib3. 2024-03-04T17:39:10.371 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-04T17:39:10.371 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-04T17:39:10.410 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T17:39:10.726 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-natsort. 2024-03-04T17:39:10.739 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-requests. 2024-03-04T17:39:10.752 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-04T17:39:10.766 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-04T17:39:10.792 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-04T17:39:10.805 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T17:39:11.078 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-logutils. 2024-03-04T17:39:11.094 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-04T17:39:11.116 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-04T17:39:11.128 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-04T17:39:11.144 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-04T17:39:11.183 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T17:39:11.415 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-04T17:39:11.442 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-04T17:39:11.478 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-04T17:39:11.481 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-04T17:39:11.505 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:11.544 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:11.785 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-mako. 2024-03-04T17:39:11.812 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-04T17:39:11.851 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T17:39:11.957 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-04T17:39:11.984 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:12.023 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:12.172 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-04T17:39:12.198 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-04T17:39:12.238 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-04T17:39:12.335 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-04T17:39:12.362 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-04T17:39:12.401 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-04T17:39:12.510 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-04T17:39:12.537 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-04T17:39:12.577 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-04T17:39:12.704 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-openssl. 2024-03-04T17:39:12.731 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-04T17:39:12.770 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-04T17:39:12.879 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-webob. 2024-03-04T17:39:12.906 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-04T17:39:12.946 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-04T17:39:13.048 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr. 2024-03-04T17:39:13.064 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:13.098 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:13.299 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-waitress. 2024-03-04T17:39:13.326 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-04T17:39:13.377 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T17:39:13.510 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mon. 2024-03-04T17:39:13.537 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:13.576 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mon (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:13.636 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-tempita. 2024-03-04T17:39:13.651 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-04T17:39:13.685 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T17:39:13.964 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-paste. 2024-03-04T17:39:13.991 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-04T17:39:14.030 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T17:39:14.180 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-osd. 2024-03-04T17:39:14.207 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:14.246 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-osd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:14.342 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-04T17:39:14.357 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-04T17:39:14.392 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T17:39:14.687 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-04T17:39:14.713 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-04T17:39:14.753 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-04T17:39:15.056 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-webtest. 2024-03-04T17:39:15.083 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-04T17:39:15.122 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-04T17:39:15.299 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph. 2024-03-04T17:39:15.326 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../066-ceph_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:15.374 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:15.434 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pecan. 2024-03-04T17:39:15.461 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-04T17:39:15.500 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T17:39:15.743 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-fuse. 2024-03-04T17:39:15.770 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:15.810 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:15.864 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-certifi. 2024-03-04T17:39:15.891 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-04T17:39:15.930 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-04T17:39:16.188 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mds. 2024-03-04T17:39:16.215 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:16.234 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-idna. 2024-03-04T17:39:16.254 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mds (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:16.260 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-04T17:39:16.292 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-04T17:39:16.604 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-urllib3. 2024-03-04T17:39:16.630 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-04T17:39:16.632 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package cephadm. 2024-03-04T17:39:16.660 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../069-cephadm_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:16.670 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T17:39:16.690 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:16.951 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-04T17:39:16.975 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-04T17:39:16.998 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-requests. 2024-03-04T17:39:17.009 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-04T17:39:17.025 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-04T17:39:17.064 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T17:39:17.359 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-04T17:39:17.386 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-04T17:39:17.426 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T17:39:17.554 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-04T17:39:17.581 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T17:39:17.620 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T17:39:17.704 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-04T17:39:17.720 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:17.754 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:18.134 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-04T17:39:18.161 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:18.200 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:18.487 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-04T17:39:18.514 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-04T17:39:18.553 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-04T17:39:18.667 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-babel. 2024-03-04T17:39:18.695 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T17:39:18.733 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T17:39:18.823 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-openssl. 2024-03-04T17:39:18.850 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-04T17:39:18.890 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-04T17:39:19.044 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-jinja2. 2024-03-04T17:39:19.072 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-04T17:39:19.111 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T17:39:19.226 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr. 2024-03-04T17:39:19.253 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:19.292 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:19.531 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-04T17:39:19.558 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:19.597 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:19.688 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mon. 2024-03-04T17:39:19.715 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:19.754 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mon (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:19.950 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-04T17:39:19.978 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-04T17:39:20.017 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-04T17:39:20.283 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-osd. 2024-03-04T17:39:20.309 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:20.319 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-routes. 2024-03-04T17:39:20.347 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-04T17:39:20.349 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-osd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:20.386 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T17:39:20.722 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-04T17:39:20.750 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:20.789 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:21.201 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-04T17:39:21.217 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-04T17:39:21.250 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T17:39:21.419 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph. 2024-03-04T17:39:21.435 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../066-ceph_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:21.478 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:21.679 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-joblib. 2024-03-04T17:39:21.706 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-04T17:39:21.737 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T17:39:21.781 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-fuse. 2024-03-04T17:39:21.807 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:21.847 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:22.099 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-04T17:39:22.126 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-04T17:39:22.165 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-04T17:39:22.342 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mds. 2024-03-04T17:39:22.369 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:22.400 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mds (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-sklearn. 2024-03-04T17:39:22.512 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-04T17:39:22.551 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T17:39:22.795 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package cephadm. 2024-03-04T17:39:22.822 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../069-cephadm_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:22.862 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:23.190 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-04T17:39:23.216 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-04T17:39:23.251 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-04T17:39:23.256 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-04T17:39:23.268 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:23.395 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:23.576 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-04T17:39:23.603 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T17:39:23.642 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T17:39:24.365 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-cachetools. 2024-03-04T17:39:24.392 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-04T17:39:24.431 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-04T17:39:24.689 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-babel. 2024-03-04T17:39:24.717 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T17:39:24.742 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-04T17:39:24.747 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T17:39:24.770 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-04T17:39:24.809 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-04T17:39:25.050 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-jinja2. 2024-03-04T17:39:25.077 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-04T17:39:25.117 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T17:39:25.137 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-04T17:39:25.165 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-04T17:39:25.204 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T17:39:25.437 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-04T17:39:25.464 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:25.490 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-rsa. 2024-03-04T17:39:25.503 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:25.517 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-04T17:39:25.556 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-04T17:39:25.815 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-04T17:39:25.842 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-04T17:39:25.844 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-google-auth. 2024-03-04T17:39:25.871 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-04T17:39:25.881 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-04T17:39:25.910 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-04T17:39:26.184 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-routes. 2024-03-04T17:39:26.211 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-04T17:39:26.229 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-04T17:39:26.251 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T17:39:26.257 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-04T17:39:26.296 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T17:39:26.596 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-04T17:39:26.607 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-websocket. 2024-03-04T17:39:26.623 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:26.635 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-04T17:39:26.663 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:26.673 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-04T17:39:27.019 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-04T17:39:27.046 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-04T17:39:27.085 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T17:39:27.116 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-04T17:39:27.131 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-04T17:39:27.166 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T17:39:27.682 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-04T17:39:27.687 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-joblib. 2024-03-04T17:39:27.699 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:27.714 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-04T17:39:27.740 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:27.745 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T17:39:28.060 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-04T17:39:28.088 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-04T17:39:28.107 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-04T17:39:28.126 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T17:39:28.134 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-04T17:39:28.190 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-04T17:39:28.471 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-04T17:39:28.493 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-sklearn. 2024-03-04T17:39:28.499 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T17:39:28.520 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-04T17:39:28.537 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T17:39:28.559 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T17:39:28.773 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package jq. 2024-03-04T17:39:28.790 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T17:39:28.823 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-04T17:39:29.059 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package socat. 2024-03-04T17:39:29.076 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-04T17:39:29.117 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-04T17:39:29.268 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-04T17:39:29.296 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:29.420 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:29.429 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package xmlstarlet. 2024-03-04T17:39:29.456 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-04T17:39:29.495 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-04T17:39:29.824 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-test. 2024-03-04T17:39:29.841 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../097-ceph-test_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:29.882 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-test (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:30.382 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-cachetools. 2024-03-04T17:39:30.409 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-04T17:39:30.440 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-04T17:39:30.710 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-04T17:39:30.738 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-04T17:39:30.776 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-04T17:39:31.105 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-04T17:39:31.132 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-04T17:39:31.172 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T17:39:31.517 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-rsa. 2024-03-04T17:39:31.544 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-04T17:39:31.583 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-04T17:39:31.904 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-google-auth. 2024-03-04T17:39:31.931 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-04T17:39:31.970 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-04T17:39:32.153 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package ceph-volume. 2024-03-04T17:39:32.181 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:32.220 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking ceph-volume (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:32.290 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-04T17:39:32.317 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-04T17:39:32.357 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T17:39:32.532 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-04T17:39:32.548 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:32.590 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcephfs-dev (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:32.626 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-websocket. 2024-03-04T17:39:32.642 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-04T17:39:32.676 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-04T17:39:32.910 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-04T17:39:32.938 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-04T17:39:32.976 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T17:39:32.980 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-04T17:39:33.007 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-04T17:39:33.046 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T17:39:33.297 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-04T17:39:33.313 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-04T17:39:33.347 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-04T17:39:33.685 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-04T17:39:33.702 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:33.718 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package nvme-cli. 2024-03-04T17:39:33.743 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:33.745 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-04T17:39:33.784 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T17:39:34.072 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-04T17:39:34.100 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-04T17:39:34.138 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T17:39:34.207 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package pkg-config. 2024-03-04T17:39:34.234 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-04T17:39:34.272 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T17:39:34.433 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-04T17:39:34.460 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T17:39:34.491 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T17:39:34.541 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-04T17:39:34.558 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-04T17:39:34.600 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-04T17:39:34.752 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package jq. 2024-03-04T17:39:34.779 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T17:39:34.838 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-04T17:39:35.070 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-04T17:39:35.072 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package socat. 2024-03-04T17:39:35.087 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-04T17:39:35.097 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-04T17:39:35.130 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-04T17:39:35.136 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-04T17:39:35.405 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pastescript. 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-04T17:39:35.455 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-04T17:39:35.476 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package xmlstarlet. 2024-03-04T17:39:35.503 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-04T17:39:35.542 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-04T17:39:35.717 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pluggy. 2024-03-04T17:39:35.745 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-04T17:39:35.792 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-04T17:39:35.888 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-test. 2024-03-04T17:39:35.904 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../097-ceph-test_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:35.946 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-test (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:36.103 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-psutil. 2024-03-04T17:39:36.131 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-04T17:39:36.169 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-04T17:39:36.522 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-py. 2024-03-04T17:39:36.550 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-04T17:39:36.588 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-04T17:39:36.900 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pygments. 2024-03-04T17:39:36.928 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-04T17:39:36.966 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T17:39:37.412 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-04T17:39:37.440 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-04T17:39:37.478 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-04T17:39:37.790 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-toml. 2024-03-04T17:39:37.817 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-04T17:39:37.856 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-04T17:39:38.127 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-pytest. 2024-03-04T17:39:38.144 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-04T17:39:38.186 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T17:39:38.351 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-volume. 2024-03-04T17:39:38.378 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:39:38.442 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-volume (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:38.523 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-simplejson. 2024-03-04T17:39:38.551 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-04T17:39:38.580 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-04T17:39:38.738 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-04T17:39:38.754 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:38.788 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libcephfs-dev (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:38.808 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-04T17:39:38.824 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-04T17:39:38.858 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-04T17:39:39.083 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-04T17:39:39.110 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-04T17:39:39.149 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T17:39:39.479 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-04T17:39:39.507 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-04T17:39:39.545 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-04T17:39:39.562 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package radosgw. 2024-03-04T17:39:39.590 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../116-radosgw_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:39.628 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking radosgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:39.867 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package nvme-cli. 2024-03-04T17:39:39.895 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-04T17:39:39.924 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T17:39:40.347 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package pkg-config. 2024-03-04T17:39:40.375 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-04T17:39:40.413 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T17:39:40.490 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package rbd-fuse. 2024-03-04T17:39:40.518 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:40.547 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking rbd-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:40.666 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-04T17:39:40.682 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-04T17:39:40.724 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-04T17:39:40.852 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-04T17:39:41.094 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T17:39:41.161 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-04T17:39:41.189 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-04T17:39:41.220 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T17:39:41.227 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-04T17:39:41.480 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pastescript. 2024-03-04T17:39:41.496 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-04T17:39:41.508 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-04T17:39:41.547 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-04T17:39:41.859 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pluggy. 2024-03-04T17:39:41.870 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-04T17:39:41.886 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-04T17:39:41.925 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-04T17:39:42.279 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-psutil. 2024-03-04T17:39:42.307 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-04T17:39:42.322 INFO:teuthology.orchestra.run.smithi047.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-04T17:39:42.345 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-04T17:39:42.397 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:42.673 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-py. 2024-03-04T17:39:42.701 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-04T17:39:42.739 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-04T17:39:42.872 INFO:teuthology.orchestra.run.smithi047.stdout:Adding system user cephadm....done 2024-03-04T17:39:43.002 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T17:39:43.068 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pygments. 2024-03-04T17:39:43.096 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-04T17:39:43.134 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T17:39:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-04T17:39:43.530 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-04T17:39:43.581 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-04T17:39:43.608 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-04T17:39:43.631 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-04T17:39:43.647 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-04T17:39:43.865 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-04T17:39:43.933 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-toml. 2024-03-04T17:39:43.961 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-04T17:39:43.991 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-04T17:39:44.125 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T17:39:44.251 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-04T17:39:44.261 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pytest. 2024-03-04T17:39:44.289 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-04T17:39:44.327 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T17:39:44.553 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T17:39:44.639 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-simplejson. 2024-03-04T17:39:44.656 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-04T17:39:44.698 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-04T17:39:44.929 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-04T17:39:44.967 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-04T17:39:44.995 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-04T17:39:45.025 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-04T17:39:45.189 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-04T17:39:45.457 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-04T17:39:45.691 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-04T17:39:45.797 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package radosgw. 2024-03-04T17:39:45.825 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../116-radosgw_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:45.863 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking radosgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:45.942 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-04T17:39:46.202 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-04T17:39:46.486 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-ceph-argparse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:46.616 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package rbd-fuse. 2024-03-04T17:39:46.632 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:39:46.674 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking rbd-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:46.812 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T17:39:46.938 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T17:39:46.995 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-04T17:39:47.064 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T17:39:47.190 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-04T17:39:47.238 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T17:39:47.364 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T17:39:47.449 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-04T17:39:47.627 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-04T17:39:47.855 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-04T17:39:47.888 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-04T17:39:47.957 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-04T17:39:47.964 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T17:39:48.087 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-04T17:39:48.411 INFO:teuthology.orchestra.run.smithi152.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-04T17:39:48.597 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T17:39:48.625 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:48.722 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-04T17:39:48.982 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T17:39:49.133 INFO:teuthology.orchestra.run.smithi152.stdout:Adding system user cephadm....done 2024-03-04T17:39:49.250 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-04T17:39:49.298 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T17:39:49.542 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up zip (3.0-12build2) ... 2024-03-04T17:39:49.574 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-04T17:39:49.677 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T17:39:49.842 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-04T17:39:49.968 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-04T17:39:50.219 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-04T17:39:50.387 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T17:39:50.613 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T17:39:50.646 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T17:39:50.739 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-04T17:39:50.764 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-04T17:39:50.865 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T17:39:51.040 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T17:39:51.149 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-04T17:39:51.401 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-04T17:39:51.542 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-04T17:39:51.652 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-04T17:39:51.785 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T17:39:51.920 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-04T17:39:52.146 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-04T17:39:52.187 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T17:39:52.389 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-04T17:39:52.488 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T17:39:52.648 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-04T17:39:52.839 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-04T17:39:52.908 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-ceph-argparse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:53.090 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-04T17:39:53.142 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T17:39:53.208 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:53.243 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T17:39:53.361 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T17:39:53.489 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-04T17:39:53.517 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-idna (3.3-1) ... 2024-03-04T17:39:53.740 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-04T17:39:53.819 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T17:39:54.150 INFO:teuthology.orchestra.run.smithi152.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-04T17:39:54.179 INFO:teuthology.orchestra.run.smithi152.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-04T17:39:54.243 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T17:39:54.369 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-04T17:39:54.728 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T17:39:54.854 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-04T17:39:55.091 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T17:39:55.131 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T17:39:55.317 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:39:55.406 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-04T17:39:55.435 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-04T17:39:55.703 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T17:39:55.716 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up zip (3.0-12build2) ... 2024-03-04T17:39:55.829 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-04T17:39:55.842 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T17:39:55.955 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T17:39:56.231 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-04T17:39:56.491 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-04T17:39:56.510 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T17:39:56.742 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:39:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T17:39:56.868 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-04T17:39:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-04T17:39:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T17:39:57.178 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-04T17:39:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-04T17:39:57.445 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-04T17:39:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-04T17:39:57.729 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-04T17:39:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T17:39:57.989 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-04T17:39:58.223 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-04T17:39:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T17:39:58.482 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T17:39:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T17:39:58.609 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-04T17:39:58.870 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-04T17:39:58.885 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T17:39:59.011 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T17:39:59.105 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-04T17:39:59.239 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:59.312 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-04T17:39:59.532 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-idna (3.3-1) ... 2024-03-04T17:39:59.630 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-04T17:39:59.874 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T17:39:59.881 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T17:39:59.982 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-04T17:40:00.091 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T17:40:00.180 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-04T17:40:00.409 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T17:40:00.519 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T17:40:00.920 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-04T17:40:01.046 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T17:40:01.205 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-04T17:40:01.298 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:40:01.331 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-04T17:40:01.432 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-04T17:40:01.590 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:40:01.692 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T17:40:01.699 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T17:40:01.817 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-04T17:40:01.943 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T17:40:02.017 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T17:40:02.236 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-04T17:40:02.310 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T17:40:02.487 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-04T17:40:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-04T17:40:02.720 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-04T17:40:02.738 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:40:02.864 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-04T17:40:02.996 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-04T17:40:03.165 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-04T17:40:03.281 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T17:40:03.416 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-04T17:40:03.634 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-04T17:40:03.701 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-04T17:40:03.952 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-04T17:40:04.011 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-04T17:40:04.203 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-04T17:40:04.312 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T17:40:04.454 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T17:40:04.580 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-04T17:40:04.665 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T17:40:04.766 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librados2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:04.873 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T17:40:04.908 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:04.999 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T17:40:05.034 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-04T17:40:05.283 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-04T17:40:05.404 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcephfs2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:05.530 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libradosstriper1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:05.575 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-04T17:40:05.656 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-04T17:40:05.826 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T17:40:05.932 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librbd1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:05.952 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-04T17:40:06.050 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-modules-core (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:06.061 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T17:40:06.134 INFO:teuthology.orchestra.run.smithi152.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-04T17:40:06.176 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:06.363 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T17:40:06.393 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T17:40:06.393 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T17:40:06.489 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T17:40:06.874 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-04T17:40:06.919 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcephfs-dev (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:07.070 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rados (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:07.133 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-04T17:40:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up librgw2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:07.251 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-04T17:40:07.364 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T17:40:07.485 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:40:07.603 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T17:40:07.912 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T17:40:08.205 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T17:40:08.481 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-04T17:40:08.607 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-04T17:40:08.659 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rbd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:08.785 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up rbd-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:08.911 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-rgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:09.037 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-cephfs (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:09.163 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:09.299 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-04T17:40:09.559 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T17:40:09.681 INFO:teuthology.orchestra.run.smithi047.stdout:Adding group ceph....done 2024-03-04T17:40:09.912 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-04T17:40:10.057 INFO:teuthology.orchestra.run.smithi047.stdout:Adding system user ceph....done 2024-03-04T17:40:10.225 INFO:teuthology.orchestra.run.smithi047.stdout:Setting system user ceph properties....done 2024-03-04T17:40:10.239 INFO:teuthology.orchestra.run.smithi047.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-04T17:40:10.288 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-04T17:40:10.382 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-04T17:40:10.573 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T17:40:10.742 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-04T17:40:10.926 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T17:40:11.052 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up librados2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:11.169 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:11.247 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-test (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:11.278 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-04T17:40:11.373 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up radosgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:11.598 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libcephfs2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:11.724 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libradosstriper1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:11.850 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-04T17:40:11.857 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T17:40:11.858 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T17:40:12.118 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up librbd1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:12.244 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-modules-core (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:12.349 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-base (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:12.370 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:12.609 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T17:40:12.610 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T17:40:12.687 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-04T17:40:13.147 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libcephfs-dev (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:13.176 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mds (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:13.272 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-rados (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:13.398 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up librgw2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:13.414 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T17:40:13.414 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T17:40:13.524 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T17:40:13.903 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:14.133 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T17:40:14.133 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T17:40:14.646 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-osd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:14.761 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-rbd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:14.887 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up rbd-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:14.905 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T17:40:14.906 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T17:40:15.013 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-rgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:15.147 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-cephfs (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:15.273 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:15.364 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:15.490 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:15.616 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mon (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:15.757 INFO:teuthology.orchestra.run.smithi152.stdout:Adding group ceph....done 2024-03-04T17:40:15.817 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T17:40:15.817 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T17:40:16.125 INFO:teuthology.orchestra.run.smithi152.stdout:Adding system user ceph....done 2024-03-04T17:40:16.293 INFO:teuthology.orchestra.run.smithi152.stdout:Setting system user ceph properties....done 2024-03-04T17:40:16.308 INFO:teuthology.orchestra.run.smithi152.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-04T17:40:16.309 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:16.444 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-04T17:40:16.469 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:16.595 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-mgr-dashboard (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:16.802 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-04T17:40:16.929 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up ceph-volume (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:17.307 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T17:40:17.323 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-test (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:17.408 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T17:40:17.449 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up radosgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:17.560 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T17:40:17.917 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T17:40:17.918 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T17:40:18.393 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-base (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:18.710 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T17:40:18.720 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-04T17:40:19.211 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mds (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:19.429 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T17:40:19.429 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T17:40:19.608 INFO:teuthology.orchestra.run.smithi047.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T17:40:19.611 DEBUG:teuthology.orchestra.run.smithi047:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-03-04T17:40:19.664 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-03-04T17:40:19.863 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2024-03-04T17:40:19.864 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2024-03-04T17:40:19.904 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout:The following additional packages will be installed: 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-03-04T17:40:20.045 INFO:teuthology.orchestra.run.smithi047.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-03-04T17:40:20.046 INFO:teuthology.orchestra.run.smithi047.stdout:Suggested packages: 2024-03-04T17:40:20.047 INFO:teuthology.orchestra.run.smithi047.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-03-04T17:40:20.047 INFO:teuthology.orchestra.run.smithi047.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-03-04T17:40:20.047 INFO:teuthology.orchestra.run.smithi047.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-03-04T17:40:20.047 INFO:teuthology.orchestra.run.smithi047.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-03-04T17:40:20.047 INFO:teuthology.orchestra.run.smithi047.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-03-04T17:40:20.084 INFO:teuthology.orchestra.run.smithi047.stdout:The following NEW packages will be installed: 2024-03-04T17:40:20.084 INFO:teuthology.orchestra.run.smithi047.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-03-04T17:40:20.084 INFO:teuthology.orchestra.run.smithi047.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-03-04T17:40:20.084 INFO:teuthology.orchestra.run.smithi047.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-03-04T17:40:20.084 INFO:teuthology.orchestra.run.smithi047.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-03-04T17:40:20.084 INFO:teuthology.orchestra.run.smithi047.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-03-04T17:40:20.084 INFO:teuthology.orchestra.run.smithi047.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-03-04T17:40:20.085 INFO:teuthology.orchestra.run.smithi047.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-03-04T17:40:20.085 INFO:teuthology.orchestra.run.smithi047.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-03-04T17:40:20.085 INFO:teuthology.orchestra.run.smithi047.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-03-04T17:40:20.085 INFO:teuthology.orchestra.run.smithi047.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-03-04T17:40:20.085 INFO:teuthology.orchestra.run.smithi047.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-03-04T17:40:20.085 INFO:teuthology.orchestra.run.smithi047.stdout: s3cmd 2024-03-04T17:40:20.122 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T17:40:20.122 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T17:40:20.290 INFO:teuthology.orchestra.run.smithi047.stdout:0 upgraded, 35 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T17:40:20.290 INFO:teuthology.orchestra.run.smithi047.stdout:Need to get 10.4 MB of archives. 2024-03-04T17:40:20.290 INFO:teuthology.orchestra.run.smithi047.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-03-04T17:40:20.290 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-03-04T17:40:20.333 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-03-04T17:40:20.333 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-03-04T17:40:20.344 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-03-04T17:40:20.428 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-03-04T17:40:20.467 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-03-04T17:40:20.471 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-03-04T17:40:20.542 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-03-04T17:40:20.593 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-03-04T17:40:20.594 INFO:teuthology.orchestra.run.smithi047.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-03-04T17:40:20.596 INFO:teuthology.orchestra.run.smithi047.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-03-04T17:40:20.630 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-osd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:20.689 INFO:teuthology.orchestra.run.smithi047.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-03-04T17:40:20.690 INFO:teuthology.orchestra.run.smithi047.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-03-04T17:40:20.722 INFO:teuthology.orchestra.run.smithi047.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-03-04T17:40:20.722 INFO:teuthology.orchestra.run.smithi047.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-03-04T17:40:20.724 INFO:teuthology.orchestra.run.smithi047.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-03-04T17:40:20.725 INFO:teuthology.orchestra.run.smithi047.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-03-04T17:40:20.725 INFO:teuthology.orchestra.run.smithi047.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-03-04T17:40:20.730 INFO:teuthology.orchestra.run.smithi047.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-03-04T17:40:20.731 INFO:teuthology.orchestra.run.smithi047.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-03-04T17:40:20.786 INFO:teuthology.orchestra.run.smithi047.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-shared-utils-java all 3.3.0-1 [149 kB] 2024-03-04T17:40:20.789 INFO:teuthology.orchestra.run.smithi047.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-03-04T17:40:20.796 INFO:teuthology.orchestra.run.smithi047.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-03-04T17:40:20.798 INFO:teuthology.orchestra.run.smithi047.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-03-04T17:40:20.798 INFO:teuthology.orchestra.run.smithi047.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-03-04T17:40:20.801 INFO:teuthology.orchestra.run.smithi047.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-03-04T17:40:20.802 INFO:teuthology.orchestra.run.smithi047.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-03-04T17:40:20.805 INFO:teuthology.orchestra.run.smithi047.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-03-04T17:40:20.832 INFO:teuthology.orchestra.run.smithi047.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-03-04T17:40:20.842 INFO:teuthology.orchestra.run.smithi047.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-03-04T17:40:20.877 INFO:teuthology.orchestra.run.smithi047.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-03-04T17:40:20.878 INFO:teuthology.orchestra.run.smithi047.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-03-04T17:40:20.903 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T17:40:20.903 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T17:40:20.952 INFO:teuthology.orchestra.run.smithi047.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-03-04T17:40:20.953 INFO:teuthology.orchestra.run.smithi047.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-03-04T17:40:20.953 INFO:teuthology.orchestra.run.smithi047.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-03-04T17:40:21.409 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 10.4 MB in 1s (12.0 MB/s) 2024-03-04T17:40:21.457 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:21.491 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libapache-pom-java. 2024-03-04T17:40:21.572 INFO:teuthology.orchestra.run.smithi047.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173563 files and directories currently installed.) 2024-03-04T17:40:21.578 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-03-04T17:40:21.583 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:21.623 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libapache-pom-java (18-1) ... 2024-03-04T17:40:21.734 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mon (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:21.884 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-03-04T17:40:21.901 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-03-04T17:40:21.943 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-03-04T17:40:21.943 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T17:40:21.943 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T17:40:22.179 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-03-04T17:40:22.196 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-03-04T17:40:22.229 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-03-04T17:40:22.409 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcdi-api-java. 2024-03-04T17:40:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-03-04T17:40:22.506 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-03-04T17:40:22.552 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:22.678 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-dashboard (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:22.792 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcommons-cli-java. 2024-03-04T17:40:22.820 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-03-04T17:40:22.838 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-volume (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:40:22.858 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-03-04T17:40:23.119 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcommons-parent-java. 2024-03-04T17:40:23.148 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-03-04T17:40:23.173 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T17:40:23.186 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-03-04T17:40:23.274 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T17:40:23.418 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T17:40:23.430 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcommons-io-java. 2024-03-04T17:40:23.447 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-03-04T17:40:23.488 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-03-04T17:40:23.707 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-03-04T17:40:23.724 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-03-04T17:40:23.757 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-03-04T17:40:24.051 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-03-04T17:40:24.068 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-03-04T17:40:24.109 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-03-04T17:40:24.455 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjsr305-java. 2024-03-04T17:40:24.483 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-03-04T17:40:24.521 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-03-04T17:40:24.561 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T17:40:24.774 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libguava-java. 2024-03-04T17:40:24.791 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-03-04T17:40:24.824 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libguava-java (29.0-6) ... 2024-03-04T17:40:25.216 INFO:teuthology.orchestra.run.smithi152.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T17:40:25.218 DEBUG:teuthology.orchestra.run.smithi152:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-03-04T17:40:25.243 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libaopalliance-java. 2024-03-04T17:40:25.271 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-03-04T17:40:25.310 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-03-04T17:40:25.322 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2024-03-04T17:40:25.524 INFO:teuthology.orchestra.run.smithi152.stdout:Building dependency tree... 2024-03-04T17:40:25.524 INFO:teuthology.orchestra.run.smithi152.stdout:Reading state information... 2024-03-04T17:40:25.562 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libguice-java. 2024-03-04T17:40:25.579 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-03-04T17:40:25.621 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-03-04T17:40:25.694 INFO:teuthology.orchestra.run.smithi152.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:40:25.694 INFO:teuthology.orchestra.run.smithi152.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:40:25.694 INFO:teuthology.orchestra.run.smithi152.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:40:25.694 INFO:teuthology.orchestra.run.smithi152.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:40:25.694 INFO:teuthology.orchestra.run.smithi152.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:40:25.695 INFO:teuthology.orchestra.run.smithi152.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:40:25.695 INFO:teuthology.orchestra.run.smithi152.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:40:25.696 INFO:teuthology.orchestra.run.smithi152.stdout:The following additional packages will be installed: 2024-03-04T17:40:25.696 INFO:teuthology.orchestra.run.smithi152.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-03-04T17:40:25.697 INFO:teuthology.orchestra.run.smithi152.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-03-04T17:40:25.698 INFO:teuthology.orchestra.run.smithi152.stdout:Suggested packages: 2024-03-04T17:40:25.698 INFO:teuthology.orchestra.run.smithi152.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-03-04T17:40:25.698 INFO:teuthology.orchestra.run.smithi152.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-03-04T17:40:25.698 INFO:teuthology.orchestra.run.smithi152.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-03-04T17:40:25.698 INFO:teuthology.orchestra.run.smithi152.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-03-04T17:40:25.698 INFO:teuthology.orchestra.run.smithi152.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-03-04T17:40:25.734 INFO:teuthology.orchestra.run.smithi152.stdout:The following NEW packages will be installed: 2024-03-04T17:40:25.734 INFO:teuthology.orchestra.run.smithi152.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-03-04T17:40:25.734 INFO:teuthology.orchestra.run.smithi152.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-03-04T17:40:25.734 INFO:teuthology.orchestra.run.smithi152.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-03-04T17:40:25.734 INFO:teuthology.orchestra.run.smithi152.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-03-04T17:40:25.734 INFO:teuthology.orchestra.run.smithi152.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-03-04T17:40:25.734 INFO:teuthology.orchestra.run.smithi152.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-03-04T17:40:25.735 INFO:teuthology.orchestra.run.smithi152.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-03-04T17:40:25.735 INFO:teuthology.orchestra.run.smithi152.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-03-04T17:40:25.735 INFO:teuthology.orchestra.run.smithi152.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-03-04T17:40:25.735 INFO:teuthology.orchestra.run.smithi152.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-03-04T17:40:25.735 INFO:teuthology.orchestra.run.smithi152.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-03-04T17:40:25.735 INFO:teuthology.orchestra.run.smithi152.stdout: s3cmd 2024-03-04T17:40:25.916 INFO:teuthology.orchestra.run.smithi152.stdout:0 upgraded, 35 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T17:40:25.916 INFO:teuthology.orchestra.run.smithi152.stdout:Need to get 10.4 MB of archives. 2024-03-04T17:40:25.916 INFO:teuthology.orchestra.run.smithi152.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-03-04T17:40:25.916 INFO:teuthology.orchestra.run.smithi152.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-03-04T17:40:25.998 INFO:teuthology.orchestra.run.smithi152.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-03-04T17:40:25.999 INFO:teuthology.orchestra.run.smithi152.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-03-04T17:40:25.999 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-03-04T17:40:26.016 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-03-04T17:40:26.026 INFO:teuthology.orchestra.run.smithi152.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-03-04T17:40:26.049 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-03-04T17:40:26.195 INFO:teuthology.orchestra.run.smithi152.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-03-04T17:40:26.252 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjansi-native-java. 2024-03-04T17:40:26.268 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-03-04T17:40:26.271 INFO:teuthology.orchestra.run.smithi152.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-03-04T17:40:26.279 INFO:teuthology.orchestra.run.smithi152.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-03-04T17:40:26.301 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-03-04T17:40:26.417 INFO:teuthology.orchestra.run.smithi152.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-03-04T17:40:26.513 INFO:teuthology.orchestra.run.smithi152.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-03-04T17:40:26.513 INFO:teuthology.orchestra.run.smithi152.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-03-04T17:40:26.517 INFO:teuthology.orchestra.run.smithi152.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-03-04T17:40:26.537 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libjansi-java. 2024-03-04T17:40:26.554 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-03-04T17:40:26.596 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libjansi-java (1.18-1) ... 2024-03-04T17:40:26.825 INFO:teuthology.orchestra.run.smithi152.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-03-04T17:40:26.848 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libmaven-parent-java. 2024-03-04T17:40:26.865 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-03-04T17:40:26.906 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-03-04T17:40:26.907 INFO:teuthology.orchestra.run.smithi152.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-03-04T17:40:27.159 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-03-04T17:40:27.176 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-03-04T17:40:27.217 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-03-04T17:40:27.425 INFO:teuthology.orchestra.run.smithi152.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-03-04T17:40:27.426 INFO:teuthology.orchestra.run.smithi152.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-03-04T17:40:27.428 INFO:teuthology.orchestra.run.smithi152.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-03-04T17:40:27.433 INFO:teuthology.orchestra.run.smithi152.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-03-04T17:40:27.434 INFO:teuthology.orchestra.run.smithi152.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-03-04T17:40:27.450 INFO:teuthology.orchestra.run.smithi152.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-03-04T17:40:27.453 INFO:teuthology.orchestra.run.smithi152.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-03-04T17:40:27.461 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-03-04T17:40:27.478 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-03-04T17:40:27.489 INFO:teuthology.orchestra.run.smithi152.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-shared-utils-java all 3.3.0-1 [149 kB] 2024-03-04T17:40:27.496 INFO:teuthology.orchestra.run.smithi152.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-03-04T17:40:27.511 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-03-04T17:40:27.661 INFO:teuthology.orchestra.run.smithi152.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-03-04T17:40:27.764 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-03-04T17:40:27.792 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-03-04T17:40:27.830 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-03-04T17:40:27.917 INFO:teuthology.orchestra.run.smithi152.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-03-04T17:40:27.923 INFO:teuthology.orchestra.run.smithi152.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-03-04T17:40:28.019 INFO:teuthology.orchestra.run.smithi152.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-03-04T17:40:28.046 INFO:teuthology.orchestra.run.smithi152.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-03-04T17:40:28.101 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-03-04T17:40:28.117 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1_all.deb ... 2024-03-04T17:40:28.127 INFO:teuthology.orchestra.run.smithi152.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-03-04T17:40:28.151 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1) ... 2024-03-04T17:40:28.227 INFO:teuthology.orchestra.run.smithi152.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-03-04T17:40:28.257 INFO:teuthology.orchestra.run.smithi152.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-03-04T17:40:28.383 INFO:teuthology.orchestra.run.smithi152.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-03-04T17:40:28.383 INFO:teuthology.orchestra.run.smithi152.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-03-04T17:40:28.451 INFO:teuthology.orchestra.run.smithi152.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-03-04T17:40:28.478 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-03-04T17:40:28.506 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-03-04T17:40:28.545 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-03-04T17:40:28.615 INFO:teuthology.orchestra.run.smithi152.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-03-04T17:40:28.698 INFO:teuthology.orchestra.run.smithi152.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-03-04T17:40:28.797 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-03-04T17:40:28.814 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-03-04T17:40:28.856 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-03-04T17:40:29.108 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-03-04T17:40:29.136 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-03-04T17:40:29.175 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-03-04T17:40:29.346 INFO:teuthology.orchestra.run.smithi152.stdout:Fetched 10.4 MB in 3s (3,273 kB/s) 2024-03-04T17:40:29.432 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libapache-pom-java. 2024-03-04T17:40:29.490 INFO:teuthology.orchestra.run.smithi152.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173563 files and directories currently installed.) 2024-03-04T17:40:29.496 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-03-04T17:40:29.512 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-03-04T17:40:29.539 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libapache-pom-java (18-1) ... 2024-03-04T17:40:29.542 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-03-04T17:40:29.580 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-03-04T17:40:29.858 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-03-04T17:40:29.886 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-03-04T17:40:29.919 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-03-04T17:40:29.924 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-03-04T17:40:29.935 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-03-04T17:40:29.977 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-03-04T17:40:30.177 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libslf4j-java. 2024-03-04T17:40:30.194 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-03-04T17:40:30.196 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-03-04T17:40:30.224 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-03-04T17:40:30.235 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-03-04T17:40:30.263 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-03-04T17:40:30.497 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libsisu-inject-java. 2024-03-04T17:40:30.526 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-03-04T17:40:30.532 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libcdi-api-java. 2024-03-04T17:40:30.548 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-03-04T17:40:30.564 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-03-04T17:40:30.590 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-03-04T17:40:30.858 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-03-04T17:40:30.859 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libcommons-cli-java. 2024-03-04T17:40:30.875 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-03-04T17:40:30.887 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-03-04T17:40:30.917 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-03-04T17:40:30.934 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-03-04T17:40:31.153 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libmaven3-core-java. 2024-03-04T17:40:31.169 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-03-04T17:40:31.178 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libcommons-parent-java. 2024-03-04T17:40:31.194 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-03-04T17:40:31.202 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-03-04T17:40:31.228 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-03-04T17:40:31.438 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libcommons-io-java. 2024-03-04T17:40:31.455 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-03-04T17:40:31.488 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-03-04T17:40:31.590 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libwagon-file-java. 2024-03-04T17:40:31.607 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-03-04T17:40:31.648 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-03-04T17:40:31.733 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-03-04T17:40:31.760 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-03-04T17:40:31.799 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-03-04T17:40:31.917 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-03-04T17:40:31.934 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-03-04T17:40:31.976 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-03-04T17:40:32.101 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-03-04T17:40:32.117 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-03-04T17:40:32.159 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-03-04T17:40:32.429 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libjsr305-java. 2024-03-04T17:40:32.456 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-03-04T17:40:32.495 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-03-04T17:40:32.604 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package maven. 2024-03-04T17:40:32.632 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-03-04T17:40:32.670 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking maven (3.6.3-5) ... 2024-03-04T17:40:32.723 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libguava-java. 2024-03-04T17:40:32.739 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-03-04T17:40:32.773 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libguava-java (29.0-6) ... 2024-03-04T17:40:32.932 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package python3-magic. 2024-03-04T17:40:32.960 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-03-04T17:40:32.998 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-03-04T17:40:33.192 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libaopalliance-java. 2024-03-04T17:40:33.219 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-03-04T17:40:33.259 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-03-04T17:40:33.301 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package s3cmd. 2024-03-04T17:40:33.329 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-03-04T17:40:33.367 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-03-04T17:40:33.494 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libguice-java. 2024-03-04T17:40:33.522 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-03-04T17:40:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-03-04T17:40:33.756 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-03-04T17:40:33.856 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-03-04T17:40:33.956 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-03-04T17:40:33.972 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-03-04T17:40:33.982 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-03-04T17:40:34.014 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-03-04T17:40:34.108 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-03-04T17:40:34.234 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-03-04T17:40:34.250 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libjansi-native-java. 2024-03-04T17:40:34.266 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-03-04T17:40:34.300 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-03-04T17:40:34.360 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-03-04T17:40:34.486 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-03-04T17:40:34.519 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libjansi-java. 2024-03-04T17:40:34.547 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-03-04T17:40:34.585 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libjansi-java (1.18-1) ... 2024-03-04T17:40:34.647 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-03-04T17:40:34.748 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-03-04T17:40:34.874 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-03-04T17:40:34.939 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libmaven-parent-java. 2024-03-04T17:40:34.967 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-03-04T17:40:35.000 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libapache-pom-java (18-1) ... 2024-03-04T17:40:35.006 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-03-04T17:40:35.134 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-03-04T17:40:35.249 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-03-04T17:40:35.260 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-03-04T17:40:35.266 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-03-04T17:40:35.299 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-03-04T17:40:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-03-04T17:40:35.512 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-03-04T17:40:35.535 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-03-04T17:40:35.551 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-03-04T17:40:35.585 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-03-04T17:40:35.638 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-03-04T17:40:35.846 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-03-04T17:40:35.862 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-03-04T17:40:35.881 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-03-04T17:40:35.913 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-03-04T17:40:35.991 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-03-04T17:40:36.142 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-03-04T17:40:36.208 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-03-04T17:40:36.236 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1_all.deb ... 2024-03-04T17:40:36.268 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libmaven-parent-java (31-2) ... 2024-03-04T17:40:36.274 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1) ... 2024-03-04T17:40:36.394 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcommons-parent-java (43-1) ... 2024-03-04T17:40:36.520 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-03-04T17:40:36.552 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-03-04T17:40:36.568 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-03-04T17:40:36.610 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-03-04T17:40:36.645 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up s3cmd (2.2.0-1) ... 2024-03-04T17:40:36.821 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-03-04T17:40:36.837 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-03-04T17:40:36.879 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-03-04T17:40:36.947 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-03-04T17:40:37.073 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-03-04T17:40:37.131 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-03-04T17:40:37.148 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-03-04T17:40:37.190 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-03-04T17:40:37.199 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libguava-java (29.0-6) ... 2024-03-04T17:40:37.325 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-03-04T17:40:37.442 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-03-04T17:40:37.451 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-03-04T17:40:37.470 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-03-04T17:40:37.509 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-03-04T17:40:37.577 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-03-04T17:40:37.703 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-03-04T17:40:37.777 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-03-04T17:40:37.794 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-03-04T17:40:37.829 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libguice-java (4.2.3-2) ... 2024-03-04T17:40:37.836 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-03-04T17:40:37.955 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libjansi-java (1.18-1) ... 2024-03-04T17:40:38.072 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libslf4j-java. 2024-03-04T17:40:38.073 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libmaven-shared-utils-java (3.3.0-1) ... 2024-03-04T17:40:38.099 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-03-04T17:40:38.138 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-03-04T17:40:38.174 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-03-04T17:40:38.291 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up maven (3.6.3-5) ... 2024-03-04T17:40:38.408 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libsisu-inject-java. 2024-03-04T17:40:38.437 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-03-04T17:40:38.475 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-03-04T17:40:38.548 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-03-04T17:40:38.612 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T17:40:38.761 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-03-04T17:40:38.777 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-03-04T17:40:38.819 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-03-04T17:40:39.072 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libmaven3-core-java. 2024-03-04T17:40:39.088 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-03-04T17:40:39.121 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-03-04T17:40:39.493 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libwagon-file-java. 2024-03-04T17:40:39.520 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-03-04T17:40:39.559 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-03-04T17:40:39.811 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-03-04T17:40:39.828 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-03-04T17:40:40.036 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-03-04T17:40:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T17:40:40.159 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:40.648 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package maven. 2024-03-04T17:40:40.676 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-03-04T17:40:40.714 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking maven (3.6.3-5) ... 2024-03-04T17:40:41.001 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-magic. 2024-03-04T17:40:41.028 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-03-04T17:40:41.067 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-03-04T17:40:41.370 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package s3cmd. 2024-03-04T17:40:41.398 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-03-04T17:40:41.436 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-03-04T17:40:41.815 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-03-04T17:40:41.916 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-03-04T17:40:42.017 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-03-04T17:40:42.143 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-03-04T17:40:42.269 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-03-04T17:40:42.395 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-03-04T17:40:42.521 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-03-04T17:40:42.632 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-03-04T17:40:42.733 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-03-04T17:40:42.834 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-03-04T17:40:42.968 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libapache-pom-java (18-1) ... 2024-03-04T17:40:43.086 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-03-04T17:40:43.186 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-03-04T17:40:43.312 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-03-04T17:40:43.438 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-03-04T17:40:43.564 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-03-04T17:40:43.824 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-03-04T17:40:43.925 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-03-04T17:40:44.026 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-03-04T17:40:44.152 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libmaven-parent-java (31-2) ... 2024-03-04T17:40:44.278 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libcommons-parent-java (43-1) ... 2024-03-04T17:40:44.404 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-03-04T17:40:44.529 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up s3cmd (2.2.0-1) ... 2024-03-04T17:40:44.831 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-03-04T17:40:44.965 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-03-04T17:40:45.091 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libguava-java (29.0-6) ... 2024-03-04T17:40:45.192 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-03-04T17:40:45.528 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-03-04T17:40:45.654 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-03-04T17:40:45.780 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-03-04T17:40:45.906 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libguice-java (4.2.3-2) ... 2024-03-04T17:40:46.032 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libjansi-java (1.18-1) ... 2024-03-04T17:40:46.158 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libmaven-shared-utils-java (3.3.0-1) ... 2024-03-04T17:40:46.267 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-03-04T17:40:46.384 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up maven (3.6.3-5) ... 2024-03-04T17:40:46.642 INFO:teuthology.orchestra.run.smithi152.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-03-04T17:40:46.713 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T17:40:47.988 INFO:teuthology.orchestra.run.smithi152.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T17:40:47.993 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:47.994 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:40:48.381 DEBUG:teuthology.orchestra.run.smithi047:> dpkg-query -W -f '${Version}' ceph 2024-03-04T17:40:48.401 INFO:teuthology.orchestra.run.smithi047.stdout:19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:40:48.402 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:40:48.402 INFO:teuthology.task.install:The correct ceph version 19.0.0-1871-g3aec3903-1jammy is installed. 2024-03-04T17:40:48.404 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:40:48.784 DEBUG:teuthology.orchestra.run.smithi152:> dpkg-query -W -f '${Version}' ceph 2024-03-04T17:40:48.803 INFO:teuthology.orchestra.run.smithi152.stdout:19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:40:48.804 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:40:48.804 INFO:teuthology.task.install:The correct ceph version 19.0.0-1871-g3aec3903-1jammy is installed. 2024-03-04T17:40:48.806 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-04T17:40:48.806 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:48.806 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T17:40:48.820 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:40:48.820 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T17:40:48.857 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-04T17:40:48.858 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:48.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T17:40:48.875 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T17:40:48.934 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:40:48.934 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T17:40:48.946 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T17:40:49.000 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-04T17:40:49.000 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:49.000 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T17:40:49.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T17:40:49.070 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:40:49.070 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T17:40:49.082 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T17:40:49.135 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-04T17:40:49.135 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:49.135 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T17:40:49.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T17:40:49.202 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:40:49.202 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T17:40:49.214 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T17:40:49.267 INFO:teuthology.run_tasks:Running task ceph... 2024-03-04T17:40:49.363 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-04T17:40:49.363 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2024-03-04T17:40:49.366 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /var/log/ceph 2024-03-04T17:40:49.380 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-04T17:40:49.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-04T17:40:49.423 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-04T17:40:49.436 INFO:tasks.ceph:Creating extra log directories... 2024-03-04T17:40:49.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-04T17:40:49.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-04T17:40:49.504 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-04T17:40:49.504 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-04T17:40:49.504 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581388', 'branch': 'wip-rgw-d4n', 'description': 'rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7581388', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rgw-d4n'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}}, 'openssl_keys': {'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}, 'root': {'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0'], 'key-type': 'rsa:4096'}}, 'rgw': {'client.0': {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'compression type': 'random', 'ec-data-pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 3, 'm': 1, 'name': 'testprofile'}, 'frontend': 'beast', 'storage classes': 'LUKEWARM, FROZEN'}, 's3tests': {'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rgw-d4n', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}}, 'owner': 'scheduled_suriarte@teuthology', 'priority': 150, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'wip-rgw-d4n', 'suite_path': '/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/pritha-srivastava/ceph.git', 'suite_sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8', 'targets': {'smithi047.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP0JC0xWUe9ufW+3yiztg7AXm5cz+sCYYvKOzGtg6sWZrVjG/ebql9HW6jJBru2iJ0jgDcg78ZnAhySMaXyig/c=', 'smithi152.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEyUHbYY+aKXXfOMbBl5Pv6EAyHHpgEvugTsX2szT1eNtIJZAZMkVOWzyqrYvW9HYqqa2VMo9BLs1tuZEe3Moo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'tox': ['client.0']}, {'workunit': {'clients': {'client.0': ['rgw/run-bucket-check.sh']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/test_rgw_s3_mp_reupload.sh']}}}, {'ragweed': {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-reshard.sh']}}}, {'s3tests-java': {'client.0': {'force-branch': 'ceph-master', 'force-repo': 'https://github.com/ceph/java_s3tests.git'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-versioning.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b1dac5519c57c269ea98a22fb7729016a1d0e4d2', 'timestamp': '2024-03-04_17:13:34', 'tube': 'smithi', 'user': 'suriarte', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-04T17:40:49.505 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-04T17:40:49.535 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-04T17:40:49.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2024-03-04T17:40:49.583 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/run/ceph 2024-03-04T17:40:49.597 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:49.597 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T17:40:49.643 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T17:40:49.643 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-03-04T17:40:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T17:40:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:40:49.691 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-03-04T17:40:49.691 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:40:49.691 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-04 17:33:51.950728522 +0000 2024-03-04T17:40:49.691 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-04 17:33:51.666731059 +0000 2024-03-04T17:40:49.692 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-04 17:33:51.666731059 +0000 2024-03-04T17:40:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-03-04T17:40:49.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T17:40:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-04T17:40:49.745 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-04T17:40:49.745 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000253779 s, 2.0 MB/s 2024-03-04T17:40:49.746 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T17:40:49.800 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-03-04T17:40:49.850 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T17:40:49.851 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:40:49.851 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 902 Links: 1 2024-03-04T17:40:49.851 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:40:49.851 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-04 17:33:52.430724234 +0000 2024-03-04T17:40:49.851 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-04 17:33:52.150726735 +0000 2024-03-04T17:40:49.851 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-04 17:33:52.150726735 +0000 2024-03-04T17:40:49.851 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-03-04T17:40:49.851 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T17:40:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-04T17:40:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-04T17:40:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000199205 s, 2.6 MB/s 2024-03-04T17:40:49.906 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T17:40:49.956 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-03-04T17:40:50.003 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T17:40:50.003 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:40:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-03-04T17:40:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:40:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-04 17:33:52.926719804 +0000 2024-03-04T17:40:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-04 17:33:52.646722305 +0000 2024-03-04T17:40:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-04 17:33:52.646722305 +0000 2024-03-04T17:40:50.003 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-03-04T17:40:50.004 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T17:40:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-04T17:40:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-04T17:40:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000305002 s, 1.7 MB/s 2024-03-04T17:40:50.059 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T17:40:50.108 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-03-04T17:40:50.154 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T17:40:50.154 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:40:50.154 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 943 Links: 1 2024-03-04T17:40:50.154 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:40:50.155 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-04 17:33:53.450715123 +0000 2024-03-04T17:40:50.155 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-04 17:33:53.158717732 +0000 2024-03-04T17:40:50.155 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-04 17:33:53.158717732 +0000 2024-03-04T17:40:50.155 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-03-04T17:40:50.155 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T17:40:50.208 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-04T17:40:50.209 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-04T17:40:50.209 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000298359 s, 1.7 MB/s 2024-03-04T17:40:50.210 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T17:40:50.261 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:40:50.261 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:40:50.261 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T17:40:50.267 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T17:40:50.267 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-03-04T17:40:50.313 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T17:40:50.313 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:40:50.313 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 873 Links: 1 2024-03-04T17:40:50.313 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:40:50.313 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-04 17:33:51.940290934 +0000 2024-03-04T17:40:50.313 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-04 17:33:51.684293221 +0000 2024-03-04T17:40:50.314 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-04 17:33:51.684293221 +0000 2024-03-04T17:40:50.314 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2024-03-04T17:40:50.314 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T17:40:50.367 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-04T17:40:50.368 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-04T17:40:50.368 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000446905 s, 1.1 MB/s 2024-03-04T17:40:50.369 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T17:40:50.419 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-03-04T17:40:50.465 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T17:40:50.465 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:40:50.465 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-03-04T17:40:50.465 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:40:50.465 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-04 17:33:52.448286397 +0000 2024-03-04T17:40:50.465 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-04 17:33:52.176288826 +0000 2024-03-04T17:40:50.465 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-04 17:33:52.176288826 +0000 2024-03-04T17:40:50.465 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2024-03-04T17:40:50.465 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T17:40:50.519 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-04T17:40:50.519 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-04T17:40:50.519 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000382625 s, 1.3 MB/s 2024-03-04T17:40:50.521 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T17:40:50.570 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-03-04T17:40:50.617 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T17:40:50.617 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:40:50.617 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-03-04T17:40:50.617 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:40:50.617 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-04 17:33:52.976281680 +0000 2024-03-04T17:40:50.617 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-04 17:33:52.696284182 +0000 2024-03-04T17:40:50.617 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-04 17:33:52.696284182 +0000 2024-03-04T17:40:50.617 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2024-03-04T17:40:50.617 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T17:40:50.671 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-04T17:40:50.671 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-04T17:40:50.671 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000412365 s, 1.2 MB/s 2024-03-04T17:40:50.673 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T17:40:50.722 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-03-04T17:40:50.769 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T17:40:50.769 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:40:50.769 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 925 Links: 1 2024-03-04T17:40:50.769 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:40:50.769 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-04 17:33:53.508276928 +0000 2024-03-04T17:40:50.769 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-04 17:33:53.256279179 +0000 2024-03-04T17:40:50.769 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-04 17:33:53.256279179 +0000 2024-03-04T17:40:50.769 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2024-03-04T17:40:50.769 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T17:40:50.823 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-04T17:40:50.823 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-04T17:40:50.823 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000351934 s, 1.5 MB/s 2024-03-04T17:40:50.825 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T17:40:50.874 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:40:50.874 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-04T17:40:50.874 INFO:tasks.ceph:Generating config... 2024-03-04T17:40:50.875 INFO:tasks.ceph:[client] debug rgw = 20 2024-03-04T17:40:50.875 INFO:tasks.ceph:[client] rgw bucket counters cache = True 2024-03-04T17:40:50.875 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2024-03-04T17:40:50.875 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2024-03-04T17:40:50.875 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-03-04T17:40:50.876 INFO:tasks.ceph:[client] rgw torrent flag = True 2024-03-04T17:40:50.876 INFO:tasks.ceph:[client] rgw user counters cache = True 2024-03-04T17:40:50.876 INFO:tasks.ceph:[client] setgroup = ceph 2024-03-04T17:40:50.876 INFO:tasks.ceph:[client] setuser = ceph 2024-03-04T17:40:50.876 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-03-04T17:40:50.876 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-03-04T17:40:50.876 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2024-03-04T17:40:50.876 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2024-03-04T17:40:50.876 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2024-03-04T17:40:50.876 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-04T17:40:50.876 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-04T17:40:50.876 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-04T17:40:50.877 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-04T17:40:50.877 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-04T17:40:50.877 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-03-04T17:40:50.877 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-04T17:40:50.877 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-04T17:40:50.877 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-04T17:40:50.877 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-04T17:40:50.877 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-04T17:40:50.877 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-04T17:40:50.877 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-04T17:40:50.877 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-04T17:40:50.877 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-04T17:40:50.878 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-04T17:40:50.878 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-04T17:40:50.878 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-04T17:40:50.878 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-04T17:40:50.878 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-04T17:40:50.878 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-03-04T17:40:50.878 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-04T17:40:50.878 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-03-04T17:40:50.878 INFO:tasks.ceph:Setting up mon.a... 2024-03-04T17:40:50.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-04T17:40:50.914 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2024-03-04T17:40:50.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-04T17:40:51.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T17:40:51.062 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.c', '[v2:172.21.15.47:3301,v1:172.21.15.47:6790]'), ('mon.b', '172.21.15.152')] 2024-03-04T17:40:51.062 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.47,[v2:172.21.15.47:3301,v1:172.21.15.47:6790],172.21.15.152', 'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-03-04T17:40:51.063 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:51.063 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-04T17:40:51.110 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.47 --addv c '[v2:172.21.15.47:3301,v1:172.21.15.47:6790]' --add b 172.21.15.152 --print /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:51.197 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:40:51.197 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:40:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T17:40:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = quincy 2024-03-04T17:40:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2024-03-04T17:40:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T17:40:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T17:40:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-03-04T17:40:51.199624+0000 2024-03-04T17:40:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 17 (quincy) 2024-03-04T17:40:51.199 INFO:teuthology.orchestra.run.smithi047.stdout:election_strategy: 1 2024-03-04T17:40:51.199 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T17:40:51.199 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T17:40:51.199 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T17:40:51.199 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-04T17:40:51.200 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-04T17:40:51.246 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 254cdce0-6dc3-4167-bfd3-d6bd440b629e... 2024-03-04T17:40:51.247 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-04T17:40:51.295 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-04T17:40:51.324 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-03-04T17:40:51.324 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-03-04T17:40:51.324 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-04T17:40:51.324 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-03-04T17:40:51.324 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.324 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.325 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-04T17:40:51.326 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,[v2:172.21.15.47:3301,v1:172.21.15.47:6790],172.21.15.152" 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: mon client directed command retry = 5 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: ms inject socket failures = 5000 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: osd heartbeat grace = 40 2024-03-04T17:40:51.327 INFO:teuthology.orchestra.run.smithi047.stdout: osd_max_pg_log_entries = 10 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: osd_min_pg_log_entries = 10 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.328 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug trim objects = true 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.329 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 1/20 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 1/20 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-03-04T17:40:51.330 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 4/10 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: osd fast shutdown = False 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: osd shutdown pgref assert = True 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-04T17:40:51.331 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-03-04T17:40:51.332 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd crush smoke test = False 2024-03-04T17:40:51.333 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: debug rgw = 20 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: rgw bucket counters cache = True 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: rgw crypt require ssl = False 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: rgw crypt s3 kms backend = testing 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: rgw torrent flag = True 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: rgw user counters cache = True 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: setgroup = ceph 2024-03-04T17:40:51.334 INFO:teuthology.orchestra.run.smithi047.stdout: setuser = ceph 2024-03-04T17:40:51.335 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-03-04T17:40:51.335 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-03-04T17:40:51.335 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-03-04T17:40:51.335 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2024-03-04T17:40:51.335 INFO:teuthology.orchestra.run.smithi152.stdout: chdir = "" 2024-03-04T17:40:51.335 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-04T17:40:51.335 INFO:teuthology.orchestra.run.smithi152.stdout: auth supported = cephx 2024-03-04T17:40:51.335 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: filestore xattr use omap = true 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: mon clock drift allowed = 1.000 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: osd crush chooseleaf type = 0 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: auth debug = true 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on old message = true 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on bug = true 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-04T17:40:51.336 INFO:teuthology.orchestra.run.smithi152.stdout: mon pg warn max object skew = 0 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: # disable pg_autoscaler by default for new pools 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default size = 2 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow primary affinity = true 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow pg remap = true 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on legacy crush tunables = false 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on crush straw calc version zero = false 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on no sortbitwise = false 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on osd down out interval zero = false 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on too few osds = false 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-04T17:40:51.337 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: mon_allow_pool_size_one = true 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: osd default data pool replay window = 5 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: mon allow pool delete = true 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: mon cluster log file level = debug 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: debug asserts on shutdown = true 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: mon health detail to clog = false 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: mon host = "172.21.15.47,[v2:172.21.15.47:3301,v1:172.21.15.47:6790],172.21.15.152" 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: mon client directed command retry = 5 2024-03-04T17:40:51.338 INFO:teuthology.orchestra.run.smithi152.stdout: ms inject socket failures = 5000 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: osd heartbeat grace = 40 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: osd_max_pg_log_entries = 10 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: osd_min_pg_log_entries = 10 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout:[osd] 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: osd journal size = 100 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub load threshold = 5.0 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub max interval = 600 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: osd mclock profile = high_recovery_ops 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.339 INFO:teuthology.orchestra.run.smithi152.stdout: osd recover clone overlap = true 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: osd recovery max chunk = 1048576 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug shutdown = true 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug op order = true 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify stray on activate = true 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug trim objects = true 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: osd open classes on start = true 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug pg log writeout = true 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: osd deep scrub update digest min age = 30 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.340 INFO:teuthology.orchestra.run.smithi152.stdout: osd map max advance = 10 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: journal zero on create = true 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: filestore ondisk finisher threads = 3 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: filestore apply finisher threads = 3 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: bdev debug aio = true 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug misdirected ops = true 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: bdev async discard = True 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: bdev enable discard = True 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore allocator = bitmap 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore block size = 96636764160 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore fsck on mount = True 2024-03-04T17:40:51.341 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluefs = 1/20 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluestore = 1/20 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: debug osd = 20 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: debug rocksdb = 4/10 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd backfillfull_ratio = 0.85 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd full ratio = 0.9 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd nearfull ratio = 0.8 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: osd failsafe full ratio = 0.95 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: osd fast shutdown = False 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: osd objectstore = bluestore 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: osd shutdown pgref assert = True 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr] 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-03-04T17:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout: debug mgr = 20 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: mgr/telemetry/nag = false 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout:[mon] 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: debug paxos = 20 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: mon data avail warn = 5 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: mon mgr mkfs grace = 240 2024-03-04T17:40:51.343 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd reporter subtree level = osd 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd prime pg temp = true 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: auth mon ticket ttl = 660 # 11m 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: auth service ticket ttl = 240 # 4m 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: # don't complain about insecure global_id in the test suite 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-04T17:40:51.344 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: # 1m isn't quite enough 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: mon_down_mkfs_grace = 2m 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_filestore_osds = false 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd crush smoke test = False 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout:[client] 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: rgw cache enabled = true 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable ops log = true 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable usage log = true 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: debug rgw = 20 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: rgw bucket counters cache = True 2024-03-04T17:40:51.345 INFO:teuthology.orchestra.run.smithi152.stdout: rgw crypt require ssl = False 2024-03-04T17:40:51.346 INFO:teuthology.orchestra.run.smithi152.stdout: rgw crypt s3 kms backend = testing 2024-03-04T17:40:51.346 INFO:teuthology.orchestra.run.smithi152.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-03-04T17:40:51.346 INFO:teuthology.orchestra.run.smithi152.stdout: rgw torrent flag = True 2024-03-04T17:40:51.346 INFO:teuthology.orchestra.run.smithi152.stdout: rgw user counters cache = True 2024-03-04T17:40:51.346 INFO:teuthology.orchestra.run.smithi152.stdout: setgroup = ceph 2024-03-04T17:40:51.346 INFO:teuthology.orchestra.run.smithi152.stdout: setuser = ceph 2024-03-04T17:40:51.346 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.a] 2024-03-04T17:40:51.346 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.c] 2024-03-04T17:40:51.346 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.b] 2024-03-04T17:40:51.348 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-04T17:40:51.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-04T17:40:51.440 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-04T17:40:51.440 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:51.440 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-04T17:40:51.446 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:51.446 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-04T17:40:51.494 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2024-03-04T17:40:51.494 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:51.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-04T17:40:51.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T17:40:51.561 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:40:51.561 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:51.606 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi152.front.sepia.ceph.com 2024-03-04T17:40:51.607 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:40:51.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-04T17:40:51.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T17:40:51.630 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:40:51.630 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:51.677 INFO:tasks.ceph:Setting up mon nodes... 2024-03-04T17:40:51.677 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-04T17:40:51.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-04T17:40:51.720 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-04T17:40:51.723 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-04T17:40:51.766 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-04T17:40:51.780 INFO:tasks.ceph:Setting up mds nodes... 2024-03-04T17:40:51.780 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-04T17:40:51.781 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-04T17:40:51.816 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-04T17:40:51.828 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-04T17:40:51.861 INFO:teuthology.orchestra.run.smithi152.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-04T17:40:51.872 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-04T17:40:51.872 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-04T17:40:51.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-04T17:40:51.885 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:40:51.885 INFO:tasks.ceph:role: osd.0 2024-03-04T17:40:51.885 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-04T17:40:51.886 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-04T17:40:51.942 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:40:51.942 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:40:51.942 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:40:51.943 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:40:51.943 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:40:51.943 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:40:51.943 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:40:51.943 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:40:51.943 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:40:51.943 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:40:51.969 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-03-04T17:40:51.971 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-03-04T17:40:51.971 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-04T17:40:52.030 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-04T17:40:52.084 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:40:52.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:40:52.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-04T17:40:52.142 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:40:52.143 INFO:tasks.ceph:role: osd.1 2024-03-04T17:40:52.143 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-04T17:40:52.143 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:40:52.198 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:40:52.224 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-03-04T17:40:52.226 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-03-04T17:40:52.226 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-04T17:40:52.292 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-04T17:40:52.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:40:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:40:52.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-04T17:40:52.402 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:40:52.402 INFO:tasks.ceph:role: osd.2 2024-03-04T17:40:52.402 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-04T17:40:52.402 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-04T17:40:52.458 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:40:52.459 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:40:52.459 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:40:52.459 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:40:52.459 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:40:52.459 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:40:52.459 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:40:52.459 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:40:52.459 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:40:52.459 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:40:52.486 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-03-04T17:40:52.488 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-03-04T17:40:52.488 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-04T17:40:52.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-04T17:40:52.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:40:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:40:52.604 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-04T17:40:52.658 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:40:52.659 INFO:tasks.ceph:role: osd.3 2024-03-04T17:40:52.659 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2024-03-04T17:40:52.659 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-04T17:40:52.714 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:40:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:40:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:40:52.714 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:40:52.715 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:40:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:40:52.715 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:40:52.715 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:40:52.715 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:40:52.715 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:40:52.741 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-03-04T17:40:52.743 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-03-04T17:40:52.744 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-04T17:40:52.804 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-04T17:40:52.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:40:52.855 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:40:52.856 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:52.962+0000 7f3fb3cfb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-04T17:40:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:52.962+0000 7f3fb3cfb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-04T17:40:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:52.962+0000 7f3fb3cfb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T17:40:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:52.962+0000 7f3fb3cfb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T17:40:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:52.962+0000 7f3fb3cfb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T17:40:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:52.962+0000 7f3fb3cfb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-04T17:40:55.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-04T17:40:55.961 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:56.038+0000 7f32e9af07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-04T17:40:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:56.038+0000 7f32e9af07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-04T17:40:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:56.038+0000 7f32e9af07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T17:40:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:56.038+0000 7f32e9af07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T17:40:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:56.038+0000 7f32e9af07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T17:40:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:56.042+0000 7f32e9af07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-04T17:40:58.999 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-04T17:40:59.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:59.134+0000 7f1ec45157c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-04T17:40:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:59.138+0000 7f1ec45157c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-04T17:40:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:59.138+0000 7f1ec45157c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T17:40:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:59.138+0000 7f1ec45157c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T17:40:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:59.138+0000 7f1ec45157c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T17:40:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:40:59.138+0000 7f1ec45157c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-04T17:41:02.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-04T17:41:02.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:41:02.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:41:02.242+0000 7f6380e6d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-04T17:41:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:41:02.246+0000 7f6380e6d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-04T17:41:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:41:02.246+0000 7f6380e6d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T17:41:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:41:02.246+0000 7f6380e6d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T17:41:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:41:02.246+0000 7f6380e6d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T17:41:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:41:02.246+0000 7f6380e6d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-04T17:41:05.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-04T17:41:05.280 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-04T17:41:05.294 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:41:05.294 INFO:tasks.ceph:role: osd.4 2024-03-04T17:41:05.294 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com 2024-03-04T17:41:05.294 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:41:05.357 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:41:05.841 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-03-04T17:41:05.843 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-03-04T17:41:05.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-04T17:41:06.007 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-04T17:41:06.017 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:41:06.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:41:06.018 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-04T17:41:06.072 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:41:06.072 INFO:tasks.ceph:role: osd.5 2024-03-04T17:41:06.072 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com 2024-03-04T17:41:06.072 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-04T17:41:06.135 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:41:06.135 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:41:06.135 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:41:06.136 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:41:06.136 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:41:06.136 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:41:06.136 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:41:06.136 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:41:06.136 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:41:06.136 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:41:06.591 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-03-04T17:41:06.593 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-03-04T17:41:06.593 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-04T17:41:06.683 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-04T17:41:06.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:41:06.694 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:41:06.694 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-04T17:41:06.748 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:41:06.748 INFO:tasks.ceph:role: osd.6 2024-03-04T17:41:06.748 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com 2024-03-04T17:41:06.748 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-04T17:41:06.812 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:41:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:41:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:41:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:41:06.812 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:41:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:41:06.812 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:41:06.812 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:41:06.813 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:41:06.813 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:41:07.278 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-03-04T17:41:07.281 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-03-04T17:41:07.281 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-04T17:41:07.411 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-04T17:41:07.422 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:41:07.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:41:07.423 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-04T17:41:07.480 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:41:07.480 INFO:tasks.ceph:role: osd.7 2024-03-04T17:41:07.480 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com 2024-03-04T17:41:07.480 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-04T17:41:07.544 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:41:07.544 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:41:07.544 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:41:07.545 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:41:07.545 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:41:07.545 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:41:07.545 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:41:07.545 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:41:07.545 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:41:07.545 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:41:08.040 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-03-04T17:41:08.042 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-03-04T17:41:08.042 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-04T17:41:08.194 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-04T17:41:08.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:41:08.206 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:41:08.206 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:41:08.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:08.315+0000 7f2e578277c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-04T17:41:08.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:08.319+0000 7f2e578277c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-04T17:41:08.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:08.319+0000 7f2e578277c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T17:41:08.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:08.319+0000 7f2e578277c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T17:41:08.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:08.319+0000 7f2e578277c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T17:41:08.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:08.319+0000 7f2e578277c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-04T17:41:11.312 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-04T17:41:11.328 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:41:11.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:11.407+0000 7f67610a87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-04T17:41:11.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:11.407+0000 7f67610a87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-04T17:41:11.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:11.407+0000 7f67610a87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T17:41:11.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:11.407+0000 7f67610a87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T17:41:11.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:11.407+0000 7f67610a87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T17:41:11.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:11.411+0000 7f67610a87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-04T17:41:14.381 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-04T17:41:14.436 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:41:14.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:14.515+0000 7f09b46127c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-04T17:41:14.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:14.515+0000 7f09b46127c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-04T17:41:14.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:14.515+0000 7f09b46127c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T17:41:14.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:14.515+0000 7f09b46127c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T17:41:14.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:14.515+0000 7f09b46127c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T17:41:14.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:14.515+0000 7f09b46127c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-04T17:41:17.488 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-04T17:41:17.545 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:41:17.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:17.623+0000 7f880b60d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-04T17:41:17.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:17.623+0000 7f880b60d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-04T17:41:17.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:17.623+0000 7f880b60d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T17:41:17.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:17.623+0000 7f880b60d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T17:41:17.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:17.623+0000 7f880b60d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T17:41:17.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-04T17:41:17.623+0000 7f880b60d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-04T17:41:20.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-04T17:41:20.661 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-04T17:41:20.661 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:20.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-04T17:41:20.676 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:20.676 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-04T17:41:20.731 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:20.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-04T17:41:20.787 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:20.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-04T17:41:20.843 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:20.843 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-04T17:41:20.899 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:20.899 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-04T17:41:20.913 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:20.913 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-04T17:41:20.967 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:20.967 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-04T17:41:21.019 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:21.019 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-04T17:41:21.071 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:21.071 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-04T17:41:21.123 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:21.123 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-04T17:41:21.129 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:21.129 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-04T17:41:21.168 INFO:tasks.ceph:Adding keys to all mons... 2024-03-04T17:41:21.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-04T17:41:21.172 DEBUG:teuthology.orchestra.run.smithi152:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-04T17:41:21.214 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.y] 2024-03-04T17:41:21.214 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCjB+ZlC8UJKxAAmHktVuEp4fOgPvbYnvr5Zg== 2024-03-04T17:41:21.214 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2024-03-04T17:41:21.214 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCkB+ZlwOOUORAARzqPOgwDsY9XUH/qwnY+Xw== 2024-03-04T17:41:21.214 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2024-03-04T17:41:21.214 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCoB+ZlxoKTAhAA2zo08hnG+l8ixYej3BJD1w== 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCrB+Zl3AVSCBAAmj1WThIWnzFOg+KZWj/rWA== 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCuB+ZlgXC+DhAAMhYDXW0cBazAqGdqgLu99A== 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCjB+ZlyqyzLRAAOfSXfNGB2FwSUxDWU1W+Tw== 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC0B+Zlo2YSExAAnLNoOrMgLyagFkXk19WpXA== 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC3B+Zl6huEGBAAGH+/Ov0jBD2F3KljD5GuSQ== 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.6] 2024-03-04T17:41:21.215 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC6B+ZlnkjuHhAAP70AImSuzNZRZRCTEsJj/g== 2024-03-04T17:41:21.216 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.7] 2024-03-04T17:41:21.216 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQC9B+Zlzo9fJRAAJK+FPU/wkyrm+PaF43VY/g== 2024-03-04T17:41:21.216 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-03-04T17:41:21.216 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCjB+Zl6dG/MBAAXxQxxoRtLGzr78nDCT/YSQ== 2024-03-04T17:41:21.216 INFO:teuthology.orchestra.run.smithi047.stdout:[client.1] 2024-03-04T17:41:21.216 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCjB+ZlhvddMxAAaZkg27BhcYMuhiIANqRHwA== 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.y] 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCjB+ZlC8UJKxAAmHktVuEp4fOgPvbYnvr5Zg== 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.0] 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCkB+ZlwOOUORAARzqPOgwDsY9XUH/qwnY+Xw== 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.1] 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCoB+ZlxoKTAhAA2zo08hnG+l8ixYej3BJD1w== 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.2] 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCrB+Zl3AVSCBAAmj1WThIWnzFOg+KZWj/rWA== 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.3] 2024-03-04T17:41:21.222 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCuB+ZlgXC+DhAAMhYDXW0cBazAqGdqgLu99A== 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.x] 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCjB+ZlyqyzLRAAOfSXfNGB2FwSUxDWU1W+Tw== 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.4] 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQC0B+Zlo2YSExAAnLNoOrMgLyagFkXk19WpXA== 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.5] 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQC3B+Zl6huEGBAAGH+/Ov0jBD2F3KljD5GuSQ== 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.6] 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQC6B+ZlnkjuHhAAP70AImSuzNZRZRCTEsJj/g== 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.7] 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQC9B+Zlzo9fJRAAJK+FPU/wkyrm+PaF43VY/g== 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-03-04T17:41:21.223 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCjB+Zl6dG/MBAAXxQxxoRtLGzr78nDCT/YSQ== 2024-03-04T17:41:21.224 INFO:teuthology.orchestra.run.smithi152.stdout:[client.1] 2024-03-04T17:41:21.224 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCjB+ZlhvddMxAAaZkg27BhcYMuhiIANqRHwA== 2024-03-04T17:41:21.224 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T17:41:21.260 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T17:41:21.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.355 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.528 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.644 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.647 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T17:41:21.696 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T17:41:21.774 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.779 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.864 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.935 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:21.952 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:22.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:22.044 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:41:22.100 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T17:41:22.132 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T17:41:22.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T17:41:22.188 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T17:41:22.270 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-04T17:41:22.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-04T17:41:22.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T17:41:22.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-04T17:41:22.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-04T17:41:22.440 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T17:41:22.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-04T17:41:22.545 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-04T17:41:22.559 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T17:41:22.654 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-04T17:41:22.664 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:41:22.666 DEBUG:teuthology.orchestra.run.smithi152:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:41:22.712 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-04T17:41:22.712 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-04T17:41:22.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-04T17:41:22.715 INFO:tasks.ceph.mon.a:Started 2024-03-04T17:41:22.715 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-04T17:41:22.715 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-04T17:41:22.717 INFO:tasks.ceph.mon.c:Started 2024-03-04T17:41:22.717 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-04T17:41:22.717 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-04T17:41:22.757 INFO:tasks.ceph.mon.b:Started 2024-03-04T17:41:22.757 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-04T17:41:22.757 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-04T17:41:22.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-04T17:41:22.761 INFO:tasks.ceph.mgr.y:Started 2024-03-04T17:41:22.762 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-04T17:41:22.762 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-04T17:41:22.764 INFO:tasks.ceph.mgr.x:Started 2024-03-04T17:41:22.764 INFO:tasks.ceph:Setting crush tunables to default 2024-03-04T17:41:22.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2024-03-04T17:41:28.470 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:28.465+0000 7f10f1162180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T17:41:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-03-04T17:41:28.987 INFO:tasks.ceph:check_enable_crimson: False 2024-03-04T17:41:28.987 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-04T17:41:28.988 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:28.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-04T17:41:29.000 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:29.000 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-04T17:41:29.005 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:29.001+0000 7f10f1162180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T17:41:29.015 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:29.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-04T17:41:29.070 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:29.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-04T17:41:29.119 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:29.120 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-04T17:41:29.133 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:29.133 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-04T17:41:29.135 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:29.131+0000 7f545a422180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T17:41:29.146 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:29.146 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-04T17:41:29.198 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-04T17:41:29.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-04T17:41:29.213 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:29.209+0000 7f10f1162180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T17:41:29.251 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new f93e9dac-c9ce-426a-bae9-61a152c5d972 0 2024-03-04T17:41:29.514 INFO:teuthology.orchestra.run.smithi152.stdout:0 2024-03-04T17:41:29.531 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new b7b27fc3-a332-4f7b-8313-b748a567a5f4 1 2024-03-04T17:41:29.649 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:29.647+0000 7f545a422180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T17:41:29.734 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-03-04T17:41:29.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 00ea9fcb-ede3-432c-8d1a-0d7de499a1d0 2 2024-03-04T17:41:29.820 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:29.817+0000 7f10f1162180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T17:41:29.847 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:29.843+0000 7f545a422180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T17:41:29.876 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:29.873+0000 7f10f1162180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T17:41:29.931 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:29.929+0000 7f10f1162180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T17:41:29.956 INFO:teuthology.orchestra.run.smithi152.stdout:2 2024-03-04T17:41:29.972 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 5bf55476-7da1-412c-9fe3-935438d942ed 3 2024-03-04T17:41:30.012 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:30.009+0000 7f10f1162180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T17:41:30.067 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:30.065+0000 7f10f1162180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T17:41:30.148 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:30.145+0000 7f10f1162180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T17:41:30.170 INFO:teuthology.orchestra.run.smithi152.stdout:3 2024-03-04T17:41:30.187 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 2cac2cbc-a66c-47ff-bc4e-2b2bb1845c60 4 2024-03-04T17:41:30.205 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:30.201+0000 7f10f1162180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T17:41:30.360 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:30.357+0000 7f10f1162180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T17:41:30.388 INFO:teuthology.orchestra.run.smithi152.stdout:4 2024-03-04T17:41:30.404 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new e2288b2f-5c2d-47a1-b05b-0fda777aa17e 5 2024-03-04T17:41:30.409 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:30.407+0000 7f545a422180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T17:41:30.425 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:30.421+0000 7f10f1162180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T17:41:30.464 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:30.463+0000 7f545a422180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T17:41:30.518 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:30.515+0000 7f545a422180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T17:41:30.597 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:30.595+0000 7f545a422180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T17:41:30.603 INFO:teuthology.orchestra.run.smithi152.stdout:5 2024-03-04T17:41:30.619 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 4a820494-23b9-436d-b624-82b428a86097 6 2024-03-04T17:41:30.651 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:30.647+0000 7f545a422180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T17:41:30.730 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:30.727+0000 7f545a422180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T17:41:30.786 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:30.783+0000 7f545a422180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T17:41:30.853 INFO:teuthology.orchestra.run.smithi152.stdout:6 2024-03-04T17:41:30.870 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 16b54ade-65dd-466f-bf28-8045dfd2398f 7 2024-03-04T17:41:30.938 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:30.935+0000 7f545a422180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T17:41:31.003 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:30.999+0000 7f545a422180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T17:41:31.072 INFO:teuthology.orchestra.run.smithi152.stdout:7 2024-03-04T17:41:31.089 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-04T17:41:31.089 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-04T17:41:31.092 INFO:tasks.ceph.osd.0:Started 2024-03-04T17:41:31.092 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-04T17:41:31.092 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-04T17:41:31.094 INFO:tasks.ceph.osd.1:Started 2024-03-04T17:41:31.095 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-04T17:41:31.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-04T17:41:31.097 INFO:tasks.ceph.osd.2:Started 2024-03-04T17:41:31.097 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-04T17:41:31.097 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-04T17:41:31.100 INFO:tasks.ceph.osd.3:Started 2024-03-04T17:41:31.100 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-04T17:41:31.100 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-04T17:41:31.102 INFO:tasks.ceph.osd.4:Started 2024-03-04T17:41:31.102 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-04T17:41:31.102 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-04T17:41:31.104 INFO:tasks.ceph.osd.5:Started 2024-03-04T17:41:31.104 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-04T17:41:31.104 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-04T17:41:31.106 INFO:tasks.ceph.osd.6:Started 2024-03-04T17:41:31.106 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-04T17:41:31.106 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-04T17:41:31.108 INFO:tasks.ceph.osd.7:Started 2024-03-04T17:41:31.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T17:41:31.156 INFO:tasks.ceph.mgr.y.smithi047.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-04T17:41:31.156 INFO:tasks.ceph.mgr.y.smithi047.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-04T17:41:31.156 INFO:tasks.ceph.mgr.y.smithi047.stderr: from numpy import show_config as show_numpy_config 2024-03-04T17:41:31.203 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:31.201+0000 7f10f1162180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T17:41:31.318 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:41:31.318 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"fsid":"254cdce0-6dc3-4167-bfd3-d6bd440b629e","created":"2024-03-04T17:41:28.149434+0000","modified":"2024-03-04T17:41:31.068483+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T17:41:31.068483+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f93e9dac-c9ce-426a-bae9-61a152c5d972","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b7b27fc3-a332-4f7b-8313-b748a567a5f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00ea9fcb-ede3-432c-8d1a-0d7de499a1d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5bf55476-7da1-412c-9fe3-935438d942ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2cac2cbc-a66c-47ff-bc4e-2b2bb1845c60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e2288b2f-5c2d-47a1-b05b-0fda777aa17e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4a820494-23b9-436d-b624-82b428a86097","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"16b54ade-65dd-466f-bf28-8045dfd2398f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:41:31.336 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-04T17:41:31.336 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-04T17:41:31.353 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:31.349+0000 7f10f1162180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T17:41:31.521 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:31.517+0000 7f10f1162180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T17:41:31.592 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:31.589+0000 7f10f1162180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T17:41:31.638 INFO:tasks.ceph.mgr.x.smithi152.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-04T17:41:31.638 INFO:tasks.ceph.mgr.x.smithi152.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-04T17:41:31.638 INFO:tasks.ceph.mgr.x.smithi152.stderr: from numpy import show_config as show_numpy_config 2024-03-04T17:41:31.720 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-03-04T17:41:31.717+0000 7f958f15a7c0 -1 Falling back to public interface 2024-03-04T17:41:31.720 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:31.719+0000 7f545a422180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T17:41:31.729 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-03-04T17:41:31.727+0000 7f040420b7c0 -1 Falling back to public interface 2024-03-04T17:41:31.730 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-03-04T17:41:31.727+0000 7fec4aaca7c0 -1 Falling back to public interface 2024-03-04T17:41:31.731 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-03-04T17:41:31.729+0000 7f1be63017c0 -1 Falling back to public interface 2024-03-04T17:41:31.737 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:31.733+0000 7f10f1162180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T17:41:31.739 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-03-04T17:41:31.737+0000 7f14f1ace7c0 -1 Falling back to public interface 2024-03-04T17:41:31.743 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-03-04T17:41:31.741+0000 7ffb52c497c0 -1 Falling back to public interface 2024-03-04T17:41:31.749 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-03-04T17:41:31.747+0000 7f6235c9d7c0 -1 Falling back to public interface 2024-03-04T17:41:31.749 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-03-04T17:41:31.747+0000 7f5e96c127c0 -1 Falling back to public interface 2024-03-04T17:41:31.906 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:31.903+0000 7f545a422180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T17:41:32.084 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:32.083+0000 7f545a422180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T17:41:32.140 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T17:41:32.153 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:32.151+0000 7f545a422180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T17:41:32.300 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:32.299+0000 7f545a422180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T17:41:32.326 INFO:teuthology.misc.health.smithi047.stdout: 2024-03-04T17:41:32.326 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":10,"fsid":"254cdce0-6dc3-4167-bfd3-d6bd440b629e","created":"2024-03-04T17:41:28.149434+0000","modified":"2024-03-04T17:41:31.068483+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T17:41:31.068483+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f93e9dac-c9ce-426a-bae9-61a152c5d972","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b7b27fc3-a332-4f7b-8313-b748a567a5f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00ea9fcb-ede3-432c-8d1a-0d7de499a1d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5bf55476-7da1-412c-9fe3-935438d942ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2cac2cbc-a66c-47ff-bc4e-2b2bb1845c60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e2288b2f-5c2d-47a1-b05b-0fda777aa17e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4a820494-23b9-436d-b624-82b428a86097","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"16b54ade-65dd-466f-bf28-8045dfd2398f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:41:32.341 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-04T17:41:32.472 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:32.469+0000 7f10f1162180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T17:41:32.551 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:32.549+0000 7f10f1162180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T17:41:32.739 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:32.737+0000 7f10f1162180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T17:41:32.816 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:32.813+0000 7f10f1162180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T17:41:32.937 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:32.935+0000 7f545a422180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T17:41:33.012 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:33.011+0000 7f545a422180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T17:41:33.121 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:33.117+0000 7f10f1162180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T17:41:33.186 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:33.181+0000 7f10f1162180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T17:41:33.192 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:33.191+0000 7f545a422180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T17:41:33.271 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:33.267+0000 7f545a422180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T17:41:33.557 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:33.555+0000 7f545a422180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T17:41:33.619 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-03-04T17:41:33.615+0000 7f545a422180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T17:41:34.223 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:34.221+0000 7f10985a9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-04T17:41:34.223 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-03-04T17:41:34.221+0000 7f10985a9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-04T17:41:35.580 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-03-04T17:41:35.579+0000 7fec4aaca7c0 -1 osd.4 0 log_to_monitors true 2024-03-04T17:41:35.591 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-03-04T17:41:35.587+0000 7f040420b7c0 -1 osd.6 0 log_to_monitors true 2024-03-04T17:41:35.597 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-03-04T17:41:35.593+0000 7f14f1ace7c0 -1 osd.3 0 log_to_monitors true 2024-03-04T17:41:35.612 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-03-04T17:41:35.609+0000 7f1be63017c0 -1 osd.1 0 log_to_monitors true 2024-03-04T17:41:35.633 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-03-04T17:41:35.631+0000 7f6235c9d7c0 -1 osd.5 0 log_to_monitors true 2024-03-04T17:41:35.645 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-03-04T17:41:35.643+0000 7f5e96c127c0 -1 osd.7 0 log_to_monitors true 2024-03-04T17:41:35.697 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-03-04T17:41:35.693+0000 7f958f15a7c0 -1 osd.0 0 log_to_monitors true 2024-03-04T17:41:35.765 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-03-04T17:41:35.761+0000 7ffb52c497c0 -1 osd.2 0 log_to_monitors true 2024-03-04T17:41:38.864 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-03-04T17:41:38.861+0000 7ffb4e3cf640 -1 osd.2 0 waiting for initial osdmap 2024-03-04T17:41:38.876 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-03-04T17:41:38.873+0000 7f14ed254640 -1 osd.3 0 waiting for initial osdmap 2024-03-04T17:41:38.891 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-03-04T17:41:38.889+0000 7ffb46a16640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:41:38.892 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-03-04T17:41:38.889+0000 7f1be1a87640 -1 osd.1 0 waiting for initial osdmap 2024-03-04T17:41:38.911 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-03-04T17:41:38.909+0000 7f14e589b640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:41:38.927 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-03-04T17:41:38.925+0000 7f1bda0ce640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:41:39.001 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-03-04T17:41:38.997+0000 7f958a8e0640 -1 osd.0 0 waiting for initial osdmap 2024-03-04T17:41:39.023 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-03-04T17:41:39.021+0000 7f9582f27640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:41:39.148 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T17:41:39.338 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-03-04T17:41:39.335+0000 7f6231423640 -1 osd.5 0 waiting for initial osdmap 2024-03-04T17:41:39.349 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-03-04T17:41:39.347+0000 7f5e92398640 -1 osd.7 0 waiting for initial osdmap 2024-03-04T17:41:39.360 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-03-04T17:41:39.359+0000 7f6229a6a640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:41:39.376 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-03-04T17:41:39.375+0000 7f5e8a9df640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:41:39.413 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-03-04T17:41:39.411+0000 7fec47a86640 -1 osd.4 0 waiting for initial osdmap 2024-03-04T17:41:39.415 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-03-04T17:41:39.411+0000 7f04011c7640 -1 osd.6 0 waiting for initial osdmap 2024-03-04T17:41:39.437 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-03-04T17:41:39.435+0000 7fec3e897640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:41:39.437 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-03-04T17:41:39.435+0000 7f03f7fd8640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:41:39.442 INFO:teuthology.misc.health.smithi047.stdout: 2024-03-04T17:41:39.442 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":12,"fsid":"254cdce0-6dc3-4167-bfd3-d6bd440b629e","created":"2024-03-04T17:41:28.149434+0000","modified":"2024-03-04T17:41:37.227951+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T17:41:31.068483+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f93e9dac-c9ce-426a-bae9-61a152c5d972","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b7b27fc3-a332-4f7b-8313-b748a567a5f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00ea9fcb-ede3-432c-8d1a-0d7de499a1d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5bf55476-7da1-412c-9fe3-935438d942ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2cac2cbc-a66c-47ff-bc4e-2b2bb1845c60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e2288b2f-5c2d-47a1-b05b-0fda777aa17e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4a820494-23b9-436d-b624-82b428a86097","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"16b54ade-65dd-466f-bf28-8045dfd2398f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:41:39.458 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-04T17:41:46.264 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T17:41:46.567 INFO:teuthology.misc.health.smithi047.stdout: 2024-03-04T17:41:46.567 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":16,"fsid":"254cdce0-6dc3-4167-bfd3-d6bd440b629e","created":"2024-03-04T17:41:28.149434+0000","modified":"2024-03-04T17:41:43.362900+0000","last_up_change":"2024-03-04T17:41:39.895032+0000","last_in_change":"2024-03-04T17:41:31.068483+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T17:41:41.256168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f93e9dac-c9ce-426a-bae9-61a152c5d972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6801","nonce":1071303395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6803","nonce":1071303395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6807","nonce":1071303395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6805","nonce":1071303395}]},"public_addr":"172.21.15.47:6801/1071303395","cluster_addr":"172.21.15.47:6803/1071303395","heartbeat_back_addr":"172.21.15.47:6807/1071303395","heartbeat_front_addr":"172.21.15.47:6805/1071303395","state":["exists","up"]},{"osd":1,"uuid":"b7b27fc3-a332-4f7b-8313-b748a567a5f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6809","nonce":450877494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6811","nonce":450877494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6815","nonce":450877494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6813","nonce":450877494}]},"public_addr":"172.21.15.47:6809/450877494","cluster_addr":"172.21.15.47:6811/450877494","heartbeat_back_addr":"172.21.15.47:6815/450877494","heartbeat_front_addr":"172.21.15.47:6813/450877494","state":["exists","up"]},{"osd":2,"uuid":"00ea9fcb-ede3-432c-8d1a-0d7de499a1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6825","nonce":2725445163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6827","nonce":2725445163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6831","nonce":2725445163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6829","nonce":2725445163}]},"public_addr":"172.21.15.47:6825/2725445163","cluster_addr":"172.21.15.47:6827/2725445163","heartbeat_back_addr":"172.21.15.47:6831/2725445163","heartbeat_front_addr":"172.21.15.47:6829/2725445163","state":["exists","up"]},{"osd":3,"uuid":"5bf55476-7da1-412c-9fe3-935438d942ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6817","nonce":851640392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6819","nonce":851640392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6823","nonce":851640392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6821","nonce":851640392}]},"public_addr":"172.21.15.47:6817/851640392","cluster_addr":"172.21.15.47:6819/851640392","heartbeat_back_addr":"172.21.15.47:6823/851640392","heartbeat_front_addr":"172.21.15.47:6821/851640392","state":["exists","up"]},{"osd":4,"uuid":"2cac2cbc-a66c-47ff-bc4e-2b2bb1845c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6809","nonce":929675582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6811","nonce":929675582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6815","nonce":929675582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6813","nonce":929675582}]},"public_addr":"172.21.15.152:6809/929675582","cluster_addr":"172.21.15.152:6811/929675582","heartbeat_back_addr":"172.21.15.152:6815/929675582","heartbeat_front_addr":"172.21.15.152:6813/929675582","state":["exists","up"]},{"osd":5,"uuid":"e2288b2f-5c2d-47a1-b05b-0fda777aa17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6817","nonce":3505984861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6820","nonce":3505984861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6826","nonce":3505984861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6823","nonce":3505984861}]},"public_addr":"172.21.15.152:6817/3505984861","cluster_addr":"172.21.15.152:6820/3505984861","heartbeat_back_addr":"172.21.15.152:6826/3505984861","heartbeat_front_addr":"172.21.15.152:6823/3505984861","state":["exists","up"]},{"osd":6,"uuid":"4a820494-23b9-436d-b624-82b428a86097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6801","nonce":4241326532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6803","nonce":4241326532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6807","nonce":4241326532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6805","nonce":4241326532}]},"public_addr":"172.21.15.152:6801/4241326532","cluster_addr":"172.21.15.152:6803/4241326532","heartbeat_back_addr":"172.21.15.152:6807/4241326532","heartbeat_front_addr":"172.21.15.152:6805/4241326532","state":["exists","up"]},{"osd":7,"uuid":"16b54ade-65dd-466f-bf28-8045dfd2398f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6821","nonce":159740591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6825","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6827","nonce":159740591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6831","nonce":159740591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6829","nonce":159740591}]},"public_addr":"172.21.15.152:6821/159740591","cluster_addr":"172.21.15.152:6827/159740591","heartbeat_back_addr":"172.21.15.152:6831/159740591","heartbeat_front_addr":"172.21.15.152:6829/159740591","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.722843+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.620410+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.790978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.570826+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.565658+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.595971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.590570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.621268+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:41:46.585 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-04T17:41:46.585 INFO:tasks.ceph:Creating RBD pool 2024-03-04T17:41:46.585 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool create rbd 8 2024-03-04T17:41:47.472 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'rbd' created 2024-03-04T17:41:47.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-03-04T17:41:48.477 INFO:teuthology.orchestra.run.smithi047.stderr:enabled application 'rbd' on pool 'rbd' 2024-03-04T17:41:48.501 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-04T17:41:48.501 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-04T17:41:48.501 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-04T17:41:48.501 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-04T17:41:48.502 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-04T17:41:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:41:48.872 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":724686481},{"type":"v1","addr":"172.21.15.47:6833","nonce":724686481}]},"active_addr":"172.21.15.47:6833/724686481","active_change":"2024-03-04T17:41:33.205896+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3444266301}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":541368674}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3882092294}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":618286623}]}]} 2024-03-04T17:41:48.876 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-04T17:41:48.876 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-04T17:41:48.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T17:41:49.190 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:41:49.191 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"fsid":"254cdce0-6dc3-4167-bfd3-d6bd440b629e","created":"2024-03-04T17:41:28.149434+0000","modified":"2024-03-04T17:41:48.465164+0000","last_up_change":"2024-03-04T17:41:39.895032+0000","last_in_change":"2024-03-04T17:41:31.068483+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T17:41:41.256168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-04T17:41:46.879280+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f93e9dac-c9ce-426a-bae9-61a152c5d972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6801","nonce":1071303395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6803","nonce":1071303395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6807","nonce":1071303395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6805","nonce":1071303395}]},"public_addr":"172.21.15.47:6801/1071303395","cluster_addr":"172.21.15.47:6803/1071303395","heartbeat_back_addr":"172.21.15.47:6807/1071303395","heartbeat_front_addr":"172.21.15.47:6805/1071303395","state":["exists","up"]},{"osd":1,"uuid":"b7b27fc3-a332-4f7b-8313-b748a567a5f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6809","nonce":450877494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6811","nonce":450877494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6815","nonce":450877494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6813","nonce":450877494}]},"public_addr":"172.21.15.47:6809/450877494","cluster_addr":"172.21.15.47:6811/450877494","heartbeat_back_addr":"172.21.15.47:6815/450877494","heartbeat_front_addr":"172.21.15.47:6813/450877494","state":["exists","up"]},{"osd":2,"uuid":"00ea9fcb-ede3-432c-8d1a-0d7de499a1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6825","nonce":2725445163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6827","nonce":2725445163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6831","nonce":2725445163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6829","nonce":2725445163}]},"public_addr":"172.21.15.47:6825/2725445163","cluster_addr":"172.21.15.47:6827/2725445163","heartbeat_back_addr":"172.21.15.47:6831/2725445163","heartbeat_front_addr":"172.21.15.47:6829/2725445163","state":["exists","up"]},{"osd":3,"uuid":"5bf55476-7da1-412c-9fe3-935438d942ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6817","nonce":851640392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6819","nonce":851640392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6823","nonce":851640392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6821","nonce":851640392}]},"public_addr":"172.21.15.47:6817/851640392","cluster_addr":"172.21.15.47:6819/851640392","heartbeat_back_addr":"172.21.15.47:6823/851640392","heartbeat_front_addr":"172.21.15.47:6821/851640392","state":["exists","up"]},{"osd":4,"uuid":"2cac2cbc-a66c-47ff-bc4e-2b2bb1845c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6809","nonce":929675582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6811","nonce":929675582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6815","nonce":929675582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6813","nonce":929675582}]},"public_addr":"172.21.15.152:6809/929675582","cluster_addr":"172.21.15.152:6811/929675582","heartbeat_back_addr":"172.21.15.152:6815/929675582","heartbeat_front_addr":"172.21.15.152:6813/929675582","state":["exists","up"]},{"osd":5,"uuid":"e2288b2f-5c2d-47a1-b05b-0fda777aa17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6817","nonce":3505984861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6820","nonce":3505984861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6826","nonce":3505984861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6823","nonce":3505984861}]},"public_addr":"172.21.15.152:6817/3505984861","cluster_addr":"172.21.15.152:6820/3505984861","heartbeat_back_addr":"172.21.15.152:6826/3505984861","heartbeat_front_addr":"172.21.15.152:6823/3505984861","state":["exists","up"]},{"osd":6,"uuid":"4a820494-23b9-436d-b624-82b428a86097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6801","nonce":4241326532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6803","nonce":4241326532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6807","nonce":4241326532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6805","nonce":4241326532}]},"public_addr":"172.21.15.152:6801/4241326532","cluster_addr":"172.21.15.152:6803/4241326532","heartbeat_back_addr":"172.21.15.152:6807/4241326532","heartbeat_front_addr":"172.21.15.152:6805/4241326532","state":["exists","up"]},{"osd":7,"uuid":"16b54ade-65dd-466f-bf28-8045dfd2398f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6821","nonce":159740591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6825","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6827","nonce":159740591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6831","nonce":159740591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6829","nonce":159740591}]},"public_addr":"172.21.15.152:6821/159740591","cluster_addr":"172.21.15.152:6827/159740591","heartbeat_back_addr":"172.21.15.152:6831/159740591","heartbeat_front_addr":"172.21.15.152:6829/159740591","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.722843+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.620410+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.790978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.570826+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.565658+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.595971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.590570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.621268+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:41:49.209 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-04T17:41:49.210 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T17:41:49.505 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:41:49.506 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"fsid":"254cdce0-6dc3-4167-bfd3-d6bd440b629e","created":"2024-03-04T17:41:28.149434+0000","modified":"2024-03-04T17:41:49.473601+0000","last_up_change":"2024-03-04T17:41:39.895032+0000","last_in_change":"2024-03-04T17:41:31.068483+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T17:41:41.256168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-04T17:41:46.879280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f93e9dac-c9ce-426a-bae9-61a152c5d972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6801","nonce":1071303395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6803","nonce":1071303395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6807","nonce":1071303395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":1071303395},{"type":"v1","addr":"172.21.15.47:6805","nonce":1071303395}]},"public_addr":"172.21.15.47:6801/1071303395","cluster_addr":"172.21.15.47:6803/1071303395","heartbeat_back_addr":"172.21.15.47:6807/1071303395","heartbeat_front_addr":"172.21.15.47:6805/1071303395","state":["exists","up"]},{"osd":1,"uuid":"b7b27fc3-a332-4f7b-8313-b748a567a5f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6809","nonce":450877494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6811","nonce":450877494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6815","nonce":450877494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":450877494},{"type":"v1","addr":"172.21.15.47:6813","nonce":450877494}]},"public_addr":"172.21.15.47:6809/450877494","cluster_addr":"172.21.15.47:6811/450877494","heartbeat_back_addr":"172.21.15.47:6815/450877494","heartbeat_front_addr":"172.21.15.47:6813/450877494","state":["exists","up"]},{"osd":2,"uuid":"00ea9fcb-ede3-432c-8d1a-0d7de499a1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6825","nonce":2725445163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6827","nonce":2725445163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6831","nonce":2725445163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2725445163},{"type":"v1","addr":"172.21.15.47:6829","nonce":2725445163}]},"public_addr":"172.21.15.47:6825/2725445163","cluster_addr":"172.21.15.47:6827/2725445163","heartbeat_back_addr":"172.21.15.47:6831/2725445163","heartbeat_front_addr":"172.21.15.47:6829/2725445163","state":["exists","up"]},{"osd":3,"uuid":"5bf55476-7da1-412c-9fe3-935438d942ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6817","nonce":851640392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6819","nonce":851640392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6823","nonce":851640392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":851640392},{"type":"v1","addr":"172.21.15.47:6821","nonce":851640392}]},"public_addr":"172.21.15.47:6817/851640392","cluster_addr":"172.21.15.47:6819/851640392","heartbeat_back_addr":"172.21.15.47:6823/851640392","heartbeat_front_addr":"172.21.15.47:6821/851640392","state":["exists","up"]},{"osd":4,"uuid":"2cac2cbc-a66c-47ff-bc4e-2b2bb1845c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6809","nonce":929675582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6811","nonce":929675582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6815","nonce":929675582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":929675582},{"type":"v1","addr":"172.21.15.152:6813","nonce":929675582}]},"public_addr":"172.21.15.152:6809/929675582","cluster_addr":"172.21.15.152:6811/929675582","heartbeat_back_addr":"172.21.15.152:6815/929675582","heartbeat_front_addr":"172.21.15.152:6813/929675582","state":["exists","up"]},{"osd":5,"uuid":"e2288b2f-5c2d-47a1-b05b-0fda777aa17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6817","nonce":3505984861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6820","nonce":3505984861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6826","nonce":3505984861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3505984861},{"type":"v1","addr":"172.21.15.152:6823","nonce":3505984861}]},"public_addr":"172.21.15.152:6817/3505984861","cluster_addr":"172.21.15.152:6820/3505984861","heartbeat_back_addr":"172.21.15.152:6826/3505984861","heartbeat_front_addr":"172.21.15.152:6823/3505984861","state":["exists","up"]},{"osd":6,"uuid":"4a820494-23b9-436d-b624-82b428a86097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6801","nonce":4241326532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6803","nonce":4241326532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6807","nonce":4241326532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":4241326532},{"type":"v1","addr":"172.21.15.152:6805","nonce":4241326532}]},"public_addr":"172.21.15.152:6801/4241326532","cluster_addr":"172.21.15.152:6803/4241326532","heartbeat_back_addr":"172.21.15.152:6807/4241326532","heartbeat_front_addr":"172.21.15.152:6805/4241326532","state":["exists","up"]},{"osd":7,"uuid":"16b54ade-65dd-466f-bf28-8045dfd2398f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6821","nonce":159740591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6825","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6827","nonce":159740591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6831","nonce":159740591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":159740591},{"type":"v1","addr":"172.21.15.152:6829","nonce":159740591}]},"public_addr":"172.21.15.152:6821/159740591","cluster_addr":"172.21.15.152:6827/159740591","heartbeat_back_addr":"172.21.15.152:6831/159740591","heartbeat_front_addr":"172.21.15.152:6829/159740591","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.722843+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.620410+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.790978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.570826+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.565658+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.595971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.590570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:41:36.621268+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:41:49.525 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-04T17:41:49.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-04T17:41:49.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-04T17:41:49.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-04T17:41:49.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-04T17:41:49.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-04T17:41:49.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-04T17:41:49.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-04T17:41:49.785 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-04T17:41:49.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-04T17:41:49.790 INFO:teuthology.orchestra.run.smithi047.stdout:55834574851 2024-03-04T17:41:49.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-04T17:41:49.801 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-04T17:41:49.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-04T17:41:49.802 INFO:teuthology.orchestra.run.smithi047.stdout:55834574851 2024-03-04T17:41:49.803 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-04T17:41:49.804 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-04T17:41:49.804 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-04T17:41:49.808 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-04T17:41:49.808 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-04T17:41:49.810 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-04T17:41:49.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-04T17:41:49.820 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-04T17:41:49.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-04T17:41:50.290 INFO:teuthology.orchestra.run.smithi047.stdout:55834574850 2024-03-04T17:41:50.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-03-04T17:41:50.316 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-04T17:41:50.326 INFO:teuthology.orchestra.run.smithi047.stdout:55834574851 2024-03-04T17:41:50.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-03-04T17:41:50.345 INFO:teuthology.orchestra.run.smithi047.stdout:55834574850 2024-03-04T17:41:50.351 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-04T17:41:50.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2024-03-04T17:41:50.354 INFO:teuthology.orchestra.run.smithi047.stdout:55834574851 2024-03-04T17:41:50.359 INFO:teuthology.orchestra.run.smithi047.stdout:55834574851 2024-03-04T17:41:50.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2024-03-04T17:41:50.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2024-03-04T17:41:50.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-03-04T17:41:50.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2024-03-04T17:41:50.388 INFO:teuthology.orchestra.run.smithi047.stdout:55834574851 2024-03-04T17:41:50.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2024-03-04T17:41:51.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-04T17:41:51.344 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-04T17:41:51.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-04T17:41:51.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-04T17:41:51.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-04T17:41:51.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-04T17:41:51.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-04T17:41:51.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-04T17:41:51.848 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-04T17:41:51.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.0 2024-03-04T17:41:51.874 DEBUG:teuthology.parallel:result is None 2024-03-04T17:41:51.901 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-04T17:41:51.902 INFO:teuthology.orchestra.run.smithi047.stdout:55834574852 2024-03-04T17:41:51.912 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-03-04T17:41:51.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.6 2024-03-04T17:41:51.925 DEBUG:teuthology.parallel:result is None 2024-03-04T17:41:51.926 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-04T17:41:51.935 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-04T17:41:51.937 INFO:teuthology.orchestra.run.smithi047.stdout:55834574853 2024-03-04T17:41:51.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.5 2024-03-04T17:41:51.941 DEBUG:teuthology.parallel:result is None 2024-03-04T17:41:51.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.3 2024-03-04T17:41:51.944 DEBUG:teuthology.parallel:result is None 2024-03-04T17:41:51.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.7 2024-03-04T17:41:51.954 DEBUG:teuthology.parallel:result is None 2024-03-04T17:41:51.957 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-03-04T17:41:51.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.1 2024-03-04T17:41:51.958 DEBUG:teuthology.parallel:result is None 2024-03-04T17:41:51.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.2 2024-03-04T17:41:51.967 DEBUG:teuthology.parallel:result is None 2024-03-04T17:41:51.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.4 2024-03-04T17:41:51.976 DEBUG:teuthology.parallel:result is None 2024-03-04T17:41:51.976 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-04T17:41:51.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T17:41:52.288 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:41:52.288 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-04T17:41:52.305 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-03-04T17:41:51.218299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235420,"kb_used_data":2592,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739300,"statfs":{"total":773094113280,"available":772853043200,"internally_reserved":0,"allocated":2654208,"data_stored":1650696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001157"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485026+0000","last_change":"2024-03-04T17:41:48.520408+0000","last_active":"2024-03-04T17:41:49.485026+0000","last_peered":"2024-03-04T17:41:49.485026+0000","last_clean":"2024-03-04T17:41:49.485026+0000","last_became_active":"2024-03-04T17:41:48.519877+0000","last_became_peered":"2024-03-04T17:41:48.519877+0000","last_unstale":"2024-03-04T17:41:49.485026+0000","last_undegraded":"2024-03-04T17:41:49.485026+0000","last_fullsized":"2024-03-04T17:41:49.485026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T05:20:55.002961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:50.140954+0000","last_change":"2024-03-04T17:41:48.521173+0000","last_active":"2024-03-04T17:41:50.140954+0000","last_peered":"2024-03-04T17:41:50.140954+0000","last_clean":"2024-03-04T17:41:50.140954+0000","last_became_active":"2024-03-04T17:41:48.520472+0000","last_became_peered":"2024-03-04T17:41:48.520472+0000","last_unstale":"2024-03-04T17:41:50.140954+0000","last_undegraded":"2024-03-04T17:41:50.140954+0000","last_fullsized":"2024-03-04T17:41:50.140954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:24:09.317449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485613+0000","last_change":"2024-03-04T17:41:48.488175+0000","last_active":"2024-03-04T17:41:49.485613+0000","last_peered":"2024-03-04T17:41:49.485613+0000","last_clean":"2024-03-04T17:41:49.485613+0000","last_became_active":"2024-03-04T17:41:48.487830+0000","last_became_peered":"2024-03-04T17:41:48.487830+0000","last_unstale":"2024-03-04T17:41:49.485613+0000","last_undegraded":"2024-03-04T17:41:49.485613+0000","last_fullsized":"2024-03-04T17:41:49.485613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:06:33.536494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.484834+0000","last_change":"2024-03-04T17:41:48.488400+0000","last_active":"2024-03-04T17:41:49.484834+0000","last_peered":"2024-03-04T17:41:49.484834+0000","last_clean":"2024-03-04T17:41:49.484834+0000","last_became_active":"2024-03-04T17:41:48.487480+0000","last_became_peered":"2024-03-04T17:41:48.487480+0000","last_unstale":"2024-03-04T17:41:49.484834+0000","last_undegraded":"2024-03-04T17:41:49.484834+0000","last_fullsized":"2024-03-04T17:41:49.484834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:45:37.069655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485209+0000","last_change":"2024-03-04T17:41:48.487473+0000","last_active":"2024-03-04T17:41:49.485209+0000","last_peered":"2024-03-04T17:41:49.485209+0000","last_clean":"2024-03-04T17:41:49.485209+0000","last_became_active":"2024-03-04T17:41:48.486975+0000","last_became_peered":"2024-03-04T17:41:48.486975+0000","last_unstale":"2024-03-04T17:41:49.485209+0000","last_undegraded":"2024-03-04T17:41:49.485209+0000","last_fullsized":"2024-03-04T17:41:49.485209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:08:42.718074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.553002+0000","last_change":"2024-03-04T17:41:48.487856+0000","last_active":"2024-03-04T17:41:49.553002+0000","last_peered":"2024-03-04T17:41:49.553002+0000","last_clean":"2024-03-04T17:41:49.553002+0000","last_became_active":"2024-03-04T17:41:48.487050+0000","last_became_peered":"2024-03-04T17:41:48.487050+0000","last_unstale":"2024-03-04T17:41:49.553002+0000","last_undegraded":"2024-03-04T17:41:49.553002+0000","last_fullsized":"2024-03-04T17:41:49.553002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:01:19.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485612+0000","last_change":"2024-03-04T17:41:48.486186+0000","last_active":"2024-03-04T17:41:49.485612+0000","last_peered":"2024-03-04T17:41:49.485612+0000","last_clean":"2024-03-04T17:41:49.485612+0000","last_became_active":"2024-03-04T17:41:48.485805+0000","last_became_peered":"2024-03-04T17:41:48.485805+0000","last_unstale":"2024-03-04T17:41:49.485612+0000","last_undegraded":"2024-03-04T17:41:49.485612+0000","last_fullsized":"2024-03-04T17:41:49.485612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:27:12.164174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485539+0000","last_change":"2024-03-04T17:41:48.520233+0000","last_active":"2024-03-04T17:41:49.485539+0000","last_peered":"2024-03-04T17:41:49.485539+0000","last_clean":"2024-03-04T17:41:49.485539+0000","last_became_active":"2024-03-04T17:41:48.519856+0000","last_became_peered":"2024-03-04T17:41:48.519856+0000","last_unstale":"2024-03-04T17:41:49.485539+0000","last_undegraded":"2024-03-04T17:41:49.485539+0000","last_fullsized":"2024-03-04T17:41:49.485539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T05:14:55.029918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:50.139489+0000","last_change":"2024-03-04T17:41:42.363279+0000","last_active":"2024-03-04T17:41:50.139489+0000","last_peered":"2024-03-04T17:41:50.139489+0000","last_clean":"2024-03-04T17:41:50.139489+0000","last_became_active":"2024-03-04T17:41:42.362513+0000","last_became_peered":"2024-03-04T17:41:42.362513+0000","last_unstale":"2024-03-04T17:41:50.139489+0000","last_undegraded":"2024-03-04T17:41:50.139489+0000","last_fullsized":"2024-03-04T17:41:50.139489+0000","mapping_epoch":14,"log_start":"16'100","ondisk_log_start":"16'100","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:41.344517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:41.344517+0000","last_clean_scrub_stamp":"2024-03-04T17:41:41.344517+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:52:30.967908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":653424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":192512,"data_stored":63056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":647676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T17:41:52.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T17:41:52.594 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:41:52.595 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-04T17:41:52.611 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-03-04T17:41:51.218299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235420,"kb_used_data":2592,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739300,"statfs":{"total":773094113280,"available":772853043200,"internally_reserved":0,"allocated":2654208,"data_stored":1650696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001157"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485026+0000","last_change":"2024-03-04T17:41:48.520408+0000","last_active":"2024-03-04T17:41:49.485026+0000","last_peered":"2024-03-04T17:41:49.485026+0000","last_clean":"2024-03-04T17:41:49.485026+0000","last_became_active":"2024-03-04T17:41:48.519877+0000","last_became_peered":"2024-03-04T17:41:48.519877+0000","last_unstale":"2024-03-04T17:41:49.485026+0000","last_undegraded":"2024-03-04T17:41:49.485026+0000","last_fullsized":"2024-03-04T17:41:49.485026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T05:20:55.002961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:50.140954+0000","last_change":"2024-03-04T17:41:48.521173+0000","last_active":"2024-03-04T17:41:50.140954+0000","last_peered":"2024-03-04T17:41:50.140954+0000","last_clean":"2024-03-04T17:41:50.140954+0000","last_became_active":"2024-03-04T17:41:48.520472+0000","last_became_peered":"2024-03-04T17:41:48.520472+0000","last_unstale":"2024-03-04T17:41:50.140954+0000","last_undegraded":"2024-03-04T17:41:50.140954+0000","last_fullsized":"2024-03-04T17:41:50.140954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:24:09.317449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485613+0000","last_change":"2024-03-04T17:41:48.488175+0000","last_active":"2024-03-04T17:41:49.485613+0000","last_peered":"2024-03-04T17:41:49.485613+0000","last_clean":"2024-03-04T17:41:49.485613+0000","last_became_active":"2024-03-04T17:41:48.487830+0000","last_became_peered":"2024-03-04T17:41:48.487830+0000","last_unstale":"2024-03-04T17:41:49.485613+0000","last_undegraded":"2024-03-04T17:41:49.485613+0000","last_fullsized":"2024-03-04T17:41:49.485613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:06:33.536494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.484834+0000","last_change":"2024-03-04T17:41:48.488400+0000","last_active":"2024-03-04T17:41:49.484834+0000","last_peered":"2024-03-04T17:41:49.484834+0000","last_clean":"2024-03-04T17:41:49.484834+0000","last_became_active":"2024-03-04T17:41:48.487480+0000","last_became_peered":"2024-03-04T17:41:48.487480+0000","last_unstale":"2024-03-04T17:41:49.484834+0000","last_undegraded":"2024-03-04T17:41:49.484834+0000","last_fullsized":"2024-03-04T17:41:49.484834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:45:37.069655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485209+0000","last_change":"2024-03-04T17:41:48.487473+0000","last_active":"2024-03-04T17:41:49.485209+0000","last_peered":"2024-03-04T17:41:49.485209+0000","last_clean":"2024-03-04T17:41:49.485209+0000","last_became_active":"2024-03-04T17:41:48.486975+0000","last_became_peered":"2024-03-04T17:41:48.486975+0000","last_unstale":"2024-03-04T17:41:49.485209+0000","last_undegraded":"2024-03-04T17:41:49.485209+0000","last_fullsized":"2024-03-04T17:41:49.485209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:08:42.718074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.553002+0000","last_change":"2024-03-04T17:41:48.487856+0000","last_active":"2024-03-04T17:41:49.553002+0000","last_peered":"2024-03-04T17:41:49.553002+0000","last_clean":"2024-03-04T17:41:49.553002+0000","last_became_active":"2024-03-04T17:41:48.487050+0000","last_became_peered":"2024-03-04T17:41:48.487050+0000","last_unstale":"2024-03-04T17:41:49.553002+0000","last_undegraded":"2024-03-04T17:41:49.553002+0000","last_fullsized":"2024-03-04T17:41:49.553002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:01:19.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485612+0000","last_change":"2024-03-04T17:41:48.486186+0000","last_active":"2024-03-04T17:41:49.485612+0000","last_peered":"2024-03-04T17:41:49.485612+0000","last_clean":"2024-03-04T17:41:49.485612+0000","last_became_active":"2024-03-04T17:41:48.485805+0000","last_became_peered":"2024-03-04T17:41:48.485805+0000","last_unstale":"2024-03-04T17:41:49.485612+0000","last_undegraded":"2024-03-04T17:41:49.485612+0000","last_fullsized":"2024-03-04T17:41:49.485612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:27:12.164174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:49.485539+0000","last_change":"2024-03-04T17:41:48.520233+0000","last_active":"2024-03-04T17:41:49.485539+0000","last_peered":"2024-03-04T17:41:49.485539+0000","last_clean":"2024-03-04T17:41:49.485539+0000","last_became_active":"2024-03-04T17:41:48.519856+0000","last_became_peered":"2024-03-04T17:41:48.519856+0000","last_unstale":"2024-03-04T17:41:49.485539+0000","last_undegraded":"2024-03-04T17:41:49.485539+0000","last_fullsized":"2024-03-04T17:41:49.485539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:47.457504+0000","last_clean_scrub_stamp":"2024-03-04T17:41:47.457504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T05:14:55.029918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:41:50.139489+0000","last_change":"2024-03-04T17:41:42.363279+0000","last_active":"2024-03-04T17:41:50.139489+0000","last_peered":"2024-03-04T17:41:50.139489+0000","last_clean":"2024-03-04T17:41:50.139489+0000","last_became_active":"2024-03-04T17:41:42.362513+0000","last_became_peered":"2024-03-04T17:41:42.362513+0000","last_unstale":"2024-03-04T17:41:50.139489+0000","last_undegraded":"2024-03-04T17:41:50.139489+0000","last_fullsized":"2024-03-04T17:41:50.139489+0000","mapping_epoch":14,"log_start":"16'100","ondisk_log_start":"16'100","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:41:41.344517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:41:41.344517+0000","last_clean_scrub_stamp":"2024-03-04T17:41:41.344517+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:52:30.967908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":653424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":192512,"data_stored":63056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":647676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":180224,"data_stored":57308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T17:41:52.612 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-04T17:41:52.612 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-04T17:41:52.612 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-04T17:41:52.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-04T17:41:52.935 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:41:52.936 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-04T17:41:52.953 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-04T17:41:52.953 INFO:teuthology.run_tasks:Running task openssl_keys... 2024-03-04T17:41:52.964 DEBUG:teuthology.task:Applying overrides for task openssl_keys: {'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}, 'root': {'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0'], 'key-type': 'rsa:4096'}} 2024-03-04T17:41:52.964 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p /home/ubuntu/cephtest/ca 2024-03-04T17:41:52.970 DEBUG:teuthology.orchestra.run.smithi047:> openssl req -x509 -nodes -newkey rsa:4096 -keyout /home/ubuntu/cephtest/ca/root.key -days 365 -out /home/ubuntu/cephtest/ca/root.crt -subj /CN=teuthology 2024-03-04T17:41:53.898 INFO:teuthology.orchestra.run.smithi047.stderr:...+...+........+....+..+.+.....+.......+..+...+...+............+.+.....+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+...+..+.+........+..........+...........+.+..+.+..+.......+.....+.......+......+...+.....+.+...+..+.+........+.+......+...+.....+.+...+..+...+...+.........+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+.+....................+.........+...+...............+...+..........+.....+....+...............+..............+.+......+..+......+......+.+........+....+........+.......+..+...+.+..+.......+...+............+.....................+.....+.+...+..+.........+.......+...+.........+.....+...+....+........+.+......+.........+....................+.........+...+...+.......+.....+...+.+...+..+....+..........................+...+..........+...+..............+...+.+...........+.+.....+.........+...+...+...+....+...+..............+.........................+...+......+.........+..+................+..............+...+......+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-03-04T17:41:54.516 INFO:teuthology.orchestra.run.smithi047.stderr:..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+..+...+....+............+...+........+...+....+...+..+...+...+....+...+...+...........+...+.+..+...+....+.....+.......+.....+.+........+.......+.....+....+.....+....+....................+......+...............+...+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.....+......+....+........+.+.....+.......+...+..+...+.......+...............+.........+...+...........+..........+..+...............+.......+..+..........+...+...+...+............+..+.+......+.....+..........+.....+...+...+...+.....................+...+............................+...+.....+....+..+.+........+...............+................+.....+.+.....+...+................+...+......+......+...+.....+.+..+.........+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-03-04T17:41:54.541 INFO:teuthology.orchestra.run.smithi047.stderr:----- 2024-03-04T17:41:54.551 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /home/ubuntu/cephtest/ca/root.crt /usr/local/share/ca-certificates/root.crt 2024-03-04T17:41:54.617 DEBUG:teuthology.orchestra.run.smithi047:> sudo update-ca-certificates 2024-03-04T17:41:54.686 INFO:teuthology.orchestra.run.smithi047.stdout:Updating certificates in /etc/ssl/certs... 2024-03-04T17:41:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL 2024-03-04T17:41:55.415 INFO:teuthology.orchestra.run.smithi047.stdout:1 added, 0 removed; done. 2024-03-04T17:41:55.425 INFO:teuthology.orchestra.run.smithi047.stdout:Running hooks in /etc/ca-certificates/update.d... 2024-03-04T17:41:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:41:57.002 INFO:teuthology.orchestra.run.smithi047.stdout:Adding debian:root.pem 2024-03-04T17:41:57.017 INFO:teuthology.orchestra.run.smithi047.stdout:done. 2024-03-04T17:41:57.017 INFO:teuthology.orchestra.run.smithi047.stdout:done. 2024-03-04T17:41:57.021 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p /home/ubuntu/cephtest/ca 2024-03-04T17:41:57.068 DEBUG:teuthology.orchestra.run.smithi047:> openssl req -nodes -newkey rsa:2048 -keyout /home/ubuntu/cephtest/ca/rgw.client.0.key -out /home/ubuntu/cephtest/ca/rgw.client.0.csr -subj /CN=smithi047.front.sepia.ceph.com 2024-03-04T17:41:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:....+.+...+......+..+....+.........+...........+.+..+.............+........+.+.........+.........+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+....+...+...+..+............+.+..................+..+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*........+...+..+.+..+.........+.........+.+...............+.....+.+..+.+............+........+......+.+.........+...........+.......+...+...+..+.........+.........+.+......+...+..+...................+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-03-04T17:41:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:...+......+.+...+..+.............+...+............+...+..+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+..+...+.......+...........+....+......+..+...+....+......+..+......+.+........+...+....+.....+.........+....+......+.....+..........+......+...+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+......+........+....+......+............+...+.....+.+.....+.+..+......+......................+.....+....+...+........+....+...+......+...............+......+.....+......+...+......+.+.....+.........+....+.........+.................+...+...+....+...+...+..+.......+...+.....+.......+........+.+..+.......+..+............................+..+..........+...+......+..+....+...............+.....+.........+.+.........+..+.......+...+.....+....+...+...............+........+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-03-04T17:41:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:----- 2024-03-04T17:41:57.326 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:41:57.326 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ca/rgw.client.0.ext 2024-03-04T17:41:57.371 DEBUG:teuthology.orchestra.run.smithi047:> openssl x509 -req -in /home/ubuntu/cephtest/ca/rgw.client.0.csr -CA /home/ubuntu/cephtest/ca/root.crt -CAkey /home/ubuntu/cephtest/ca/root.key -CAcreateserial -out /home/ubuntu/cephtest/ca/rgw.client.0.crt -days 365 -sha256 -extfile /home/ubuntu/cephtest/ca/rgw.client.0.ext 2024-03-04T17:41:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:Certificate request self-signature ok 2024-03-04T17:41:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:subject=CN = smithi047.front.sepia.ceph.com 2024-03-04T17:41:57.438 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/ca/rgw.client.0.csr /home/ubuntu/cephtest/ca/root.srl /home/ubuntu/cephtest/ca/rgw.client.0.ext 2024-03-04T17:41:57.487 DEBUG:teuthology.orchestra.run.smithi047:> openssl verify -CAfile /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/rgw.client.0.crt 2024-03-04T17:41:57.541 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/ca/rgw.client.0.crt: OK 2024-03-04T17:41:57.541 DEBUG:teuthology.orchestra.run.smithi047:> cat /home/ubuntu/cephtest/ca/rgw.client.0.key >> /home/ubuntu/cephtest/ca/rgw.client.0.crt 2024-03-04T17:41:57.588 INFO:teuthology.run_tasks:Running task rgw... 2024-03-04T17:41:57.604 DEBUG:tasks.rgw:config is {'client.0': {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2024-03-04T17:41:57.604 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-03-04T17:41:57.604 INFO:tasks.rgw:Creating data pools 2024-03-04T17:41:57.604 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-03-04T17:41:57.604 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=3 m=1 name=testprofile --cluster ceph 2024-03-04T17:41:58.536 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-03-04T17:42:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'default.rgw.buckets.data' created 2024-03-04T17:42:00.562 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-03-04T17:42:01.528 INFO:teuthology.orchestra.run.smithi047.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-03-04T17:42:01.595 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-03-04T17:42:02.577 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'default.rgw.buckets.index' created 2024-03-04T17:42:02.616 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-03-04T17:42:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-03-04T17:42:03.624 DEBUG:tasks.rgw:Pools created 2024-03-04T17:42:03.624 INFO:tasks.rgw:Configuring compression type = random 2024-03-04T17:42:03.624 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-03-04T17:42:03.624 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-03-04T17:42:03.625 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-03-04T17:42:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:42:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:42:05.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.712+0000 7f45b84ef640 0 -- 172.21.15.47:0/2448346893 >> [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] conn(0x7f448c065850 msgr2=0x7f448c067d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:42:05.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.720+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.720+0000 7f45b84ef640 0 -- 172.21.15.47:0/2163533479 >> [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] conn(0x7f447c065940 msgr2=0x7f447c067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:42:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.720+0000 7f45ba1d6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.720+0000 7f45ba1d6980 20 realm 2024-03-04T17:42:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.720+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.724+0000 7f45ba1d6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.724+0000 7f45ba1d6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:42:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.724+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.724+0000 7f45ba1d6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.724+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.724+0000 7f45ba1d6980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:42:05.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.724+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:05.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.724+0000 7f45ba1d6980 20 rados_obj.operate() r=0 bl.length=875 2024-03-04T17:42:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.724+0000 7f45ba1d6980 20 searching for the correct realm 2024-03-04T17:42:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:42:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:42:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:42:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:42:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:42:05.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:42:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:42:05.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 rados_obj.operate() r=0 bl.length=398 2024-03-04T17:42:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 zone default found 2024-03-04T17:42:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 4 Realm: () 2024-03-04T17:42:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:42:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:42:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:42:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 zonegroup default 2024-03-04T17:42:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.748+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:05.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.752+0000 7f45ba1d6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:05.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:05.752+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:07.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:07.660+0000 7f45ba1d6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:07.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:07.660+0000 7f45ba1d6980 20 rados->read ofs=0 len=0 2024-03-04T17:42:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:07.692+0000 7f45ba1d6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:07.692+0000 7f45ba1d6980 20 started sync module instance, tier type = 2024-03-04T17:42:07.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:07.692+0000 7f45ba1d6980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:42:09.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 add_watcher() i=0 2024-03-04T17:42:09.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 add_watcher() i=1 2024-03-04T17:42:09.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 add_watcher() i=2 2024-03-04T17:42:09.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 add_watcher() i=3 2024-03-04T17:42:09.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 add_watcher() i=4 2024-03-04T17:42:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 add_watcher() i=5 2024-03-04T17:42:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 add_watcher() i=6 2024-03-04T17:42:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 add_watcher() i=7 2024-03-04T17:42:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 2 all 8 watchers are set, enabling cache 2024-03-04T17:42:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2024-03-04T17:42:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2024-03-04T17:42:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.724+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2024-03-04T17:42:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2024-03-04T17:42:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2024-03-04T17:42:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2024-03-04T17:42:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2024-03-04T17:42:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2024-03-04T17:42:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2024-03-04T17:42:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.4 2024-03-04T17:42:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.4 does not exist 2024-03-04T17:42:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.5 2024-03-04T17:42:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.728+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.5 does not exist 2024-03-04T17:42:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.6 2024-03-04T17:42:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.6 does not exist 2024-03-04T17:42:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.7 2024-03-04T17:42:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.7 does not exist 2024-03-04T17:42:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.8 2024-03-04T17:42:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.8 does not exist 2024-03-04T17:42:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.9 2024-03-04T17:42:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.9 does not exist 2024-03-04T17:42:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.10 2024-03-04T17:42:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.10 does not exist 2024-03-04T17:42:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.11 2024-03-04T17:42:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.11 does not exist 2024-03-04T17:42:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.12 2024-03-04T17:42:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.12 does not exist 2024-03-04T17:42:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.13 2024-03-04T17:42:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.732+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.13 does not exist 2024-03-04T17:42:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.14 2024-03-04T17:42:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.14 does not exist 2024-03-04T17:42:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.15 2024-03-04T17:42:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.15 does not exist 2024-03-04T17:42:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.16 2024-03-04T17:42:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.16 does not exist 2024-03-04T17:42:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.17 2024-03-04T17:42:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.17 does not exist 2024-03-04T17:42:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.18 2024-03-04T17:42:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.18 does not exist 2024-03-04T17:42:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.19 2024-03-04T17:42:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.736+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.19 does not exist 2024-03-04T17:42:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.20 2024-03-04T17:42:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.20 does not exist 2024-03-04T17:42:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.21 2024-03-04T17:42:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.21 does not exist 2024-03-04T17:42:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.22 2024-03-04T17:42:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.22 does not exist 2024-03-04T17:42:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.23 2024-03-04T17:42:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.23 does not exist 2024-03-04T17:42:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.24 2024-03-04T17:42:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.24 does not exist 2024-03-04T17:42:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.25 2024-03-04T17:42:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.740+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.25 does not exist 2024-03-04T17:42:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.26 2024-03-04T17:42:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.26 does not exist 2024-03-04T17:42:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.27 2024-03-04T17:42:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.27 does not exist 2024-03-04T17:42:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.28 2024-03-04T17:42:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.28 does not exist 2024-03-04T17:42:09.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.29 2024-03-04T17:42:09.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.29 does not exist 2024-03-04T17:42:09.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.30 2024-03-04T17:42:09.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.30 does not exist 2024-03-04T17:42:09.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.31 2024-03-04T17:42:09.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.744+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.31 does not exist 2024-03-04T17:42:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.32 2024-03-04T17:42:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.32 does not exist 2024-03-04T17:42:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.33 2024-03-04T17:42:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.33 does not exist 2024-03-04T17:42:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.34 2024-03-04T17:42:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.34 does not exist 2024-03-04T17:42:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.35 2024-03-04T17:42:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.35 does not exist 2024-03-04T17:42:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.36 2024-03-04T17:42:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.36 does not exist 2024-03-04T17:42:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.37 2024-03-04T17:42:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.748+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.37 does not exist 2024-03-04T17:42:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.38 2024-03-04T17:42:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.38 does not exist 2024-03-04T17:42:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.39 2024-03-04T17:42:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.39 does not exist 2024-03-04T17:42:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.40 2024-03-04T17:42:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.40 does not exist 2024-03-04T17:42:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.41 2024-03-04T17:42:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.41 does not exist 2024-03-04T17:42:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.42 2024-03-04T17:42:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.42 does not exist 2024-03-04T17:42:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.43 2024-03-04T17:42:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.43 does not exist 2024-03-04T17:42:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.44 2024-03-04T17:42:09.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.752+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.44 does not exist 2024-03-04T17:42:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.45 2024-03-04T17:42:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.45 does not exist 2024-03-04T17:42:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.46 2024-03-04T17:42:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.46 does not exist 2024-03-04T17:42:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.47 2024-03-04T17:42:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.47 does not exist 2024-03-04T17:42:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.48 2024-03-04T17:42:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.48 does not exist 2024-03-04T17:42:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.49 2024-03-04T17:42:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.49 does not exist 2024-03-04T17:42:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.50 2024-03-04T17:42:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.756+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.50 does not exist 2024-03-04T17:42:09.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.51 2024-03-04T17:42:09.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.51 does not exist 2024-03-04T17:42:09.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.52 2024-03-04T17:42:09.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.52 does not exist 2024-03-04T17:42:09.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.53 2024-03-04T17:42:09.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.53 does not exist 2024-03-04T17:42:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.54 2024-03-04T17:42:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.54 does not exist 2024-03-04T17:42:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.55 2024-03-04T17:42:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.55 does not exist 2024-03-04T17:42:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.56 2024-03-04T17:42:09.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.760+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.56 does not exist 2024-03-04T17:42:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.57 2024-03-04T17:42:09.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.57 does not exist 2024-03-04T17:42:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.58 2024-03-04T17:42:09.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.58 does not exist 2024-03-04T17:42:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.59 2024-03-04T17:42:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.59 does not exist 2024-03-04T17:42:09.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.60 2024-03-04T17:42:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.60 does not exist 2024-03-04T17:42:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.61 2024-03-04T17:42:09.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.764+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.61 does not exist 2024-03-04T17:42:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.62 2024-03-04T17:42:09.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.62 does not exist 2024-03-04T17:42:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.63 2024-03-04T17:42:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.63 does not exist 2024-03-04T17:42:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.64 2024-03-04T17:42:09.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.64 does not exist 2024-03-04T17:42:09.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.65 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.65 does not exist 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.66 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.768+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.66 does not exist 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.67 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.67 does not exist 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.68 2024-03-04T17:42:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.68 does not exist 2024-03-04T17:42:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.69 2024-03-04T17:42:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.69 does not exist 2024-03-04T17:42:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.70 2024-03-04T17:42:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.70 does not exist 2024-03-04T17:42:09.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.71 2024-03-04T17:42:09.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.71 does not exist 2024-03-04T17:42:09.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.72 2024-03-04T17:42:09.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.772+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.72 does not exist 2024-03-04T17:42:09.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.73 2024-03-04T17:42:09.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.73 does not exist 2024-03-04T17:42:09.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.74 2024-03-04T17:42:09.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.74 does not exist 2024-03-04T17:42:09.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.75 2024-03-04T17:42:09.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.75 does not exist 2024-03-04T17:42:09.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.76 2024-03-04T17:42:09.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.76 does not exist 2024-03-04T17:42:09.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.77 2024-03-04T17:42:09.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.776+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.77 does not exist 2024-03-04T17:42:09.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.78 2024-03-04T17:42:09.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.78 does not exist 2024-03-04T17:42:09.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.79 2024-03-04T17:42:09.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.79 does not exist 2024-03-04T17:42:09.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.80 2024-03-04T17:42:09.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.80 does not exist 2024-03-04T17:42:09.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.81 2024-03-04T17:42:09.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.81 does not exist 2024-03-04T17:42:09.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.82 2024-03-04T17:42:09.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.82 does not exist 2024-03-04T17:42:09.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.83 2024-03-04T17:42:09.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.780+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.83 does not exist 2024-03-04T17:42:09.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.84 2024-03-04T17:42:09.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.84 does not exist 2024-03-04T17:42:09.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.85 2024-03-04T17:42:09.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.85 does not exist 2024-03-04T17:42:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.86 2024-03-04T17:42:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.86 does not exist 2024-03-04T17:42:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.87 2024-03-04T17:42:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.87 does not exist 2024-03-04T17:42:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.88 2024-03-04T17:42:09.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.88 does not exist 2024-03-04T17:42:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.89 2024-03-04T17:42:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.784+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.89 does not exist 2024-03-04T17:42:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.90 2024-03-04T17:42:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.90 does not exist 2024-03-04T17:42:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.91 2024-03-04T17:42:09.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.91 does not exist 2024-03-04T17:42:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.92 2024-03-04T17:42:09.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.92 does not exist 2024-03-04T17:42:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.93 2024-03-04T17:42:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.93 does not exist 2024-03-04T17:42:09.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.94 2024-03-04T17:42:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.94 does not exist 2024-03-04T17:42:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.95 2024-03-04T17:42:09.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.788+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.95 does not exist 2024-03-04T17:42:09.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.96 2024-03-04T17:42:09.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.96 does not exist 2024-03-04T17:42:09.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.97 2024-03-04T17:42:09.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.97 does not exist 2024-03-04T17:42:09.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.98 2024-03-04T17:42:09.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.98 does not exist 2024-03-04T17:42:09.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.99 2024-03-04T17:42:09.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.99 does not exist 2024-03-04T17:42:09.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.100 2024-03-04T17:42:09.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.792+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.100 does not exist 2024-03-04T17:42:09.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.101 2024-03-04T17:42:09.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.101 does not exist 2024-03-04T17:42:09.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.102 2024-03-04T17:42:09.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.102 does not exist 2024-03-04T17:42:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.103 2024-03-04T17:42:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.103 does not exist 2024-03-04T17:42:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.104 2024-03-04T17:42:09.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.104 does not exist 2024-03-04T17:42:09.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.105 2024-03-04T17:42:09.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.105 does not exist 2024-03-04T17:42:09.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.106 2024-03-04T17:42:09.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.796+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.106 does not exist 2024-03-04T17:42:09.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.107 2024-03-04T17:42:09.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.107 does not exist 2024-03-04T17:42:09.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.108 2024-03-04T17:42:09.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.108 does not exist 2024-03-04T17:42:09.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.109 2024-03-04T17:42:09.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.109 does not exist 2024-03-04T17:42:09.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.110 2024-03-04T17:42:09.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.110 does not exist 2024-03-04T17:42:09.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.111 2024-03-04T17:42:09.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.111 does not exist 2024-03-04T17:42:09.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.112 2024-03-04T17:42:09.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.800+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.112 does not exist 2024-03-04T17:42:09.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.113 2024-03-04T17:42:09.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.113 does not exist 2024-03-04T17:42:09.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.114 2024-03-04T17:42:09.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.114 does not exist 2024-03-04T17:42:09.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.115 2024-03-04T17:42:09.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.115 does not exist 2024-03-04T17:42:09.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.116 2024-03-04T17:42:09.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.116 does not exist 2024-03-04T17:42:09.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.117 2024-03-04T17:42:09.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.117 does not exist 2024-03-04T17:42:09.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.118 2024-03-04T17:42:09.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.804+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.118 does not exist 2024-03-04T17:42:09.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.119 2024-03-04T17:42:09.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.119 does not exist 2024-03-04T17:42:09.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.120 2024-03-04T17:42:09.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.120 does not exist 2024-03-04T17:42:09.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.121 2024-03-04T17:42:09.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.121 does not exist 2024-03-04T17:42:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.122 2024-03-04T17:42:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.122 does not exist 2024-03-04T17:42:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.123 2024-03-04T17:42:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.123 does not exist 2024-03-04T17:42:09.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.124 2024-03-04T17:42:09.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.808+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.124 does not exist 2024-03-04T17:42:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.125 2024-03-04T17:42:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.125 does not exist 2024-03-04T17:42:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.126 2024-03-04T17:42:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.126 does not exist 2024-03-04T17:42:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.127 2024-03-04T17:42:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.127 does not exist 2024-03-04T17:42:09.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.812+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1211 entering 2024-03-04T17:42:09.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.816+0000 7f45ba1d6980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:42:09.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.824+0000 7f4469fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:42:09.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.824+0000 7f45ba1d6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:42:09.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:09.824+0000 7f45ba1d6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:42:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.688+0000 7f45ba1d6980 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2024-03-04T17:42:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.688+0000 7f45ba1d6980 5 note: GC not initialized 2024-03-04T17:42:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.700+0000 7f4468fe1640 20 reqs_thread_entry: start 2024-03-04T17:42:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.700+0000 7f45ba1d6980 20 init_complete bucket index max shards: 11 2024-03-04T17:42:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.700+0000 7f45ba1d6980 20 Filter name: none 2024-03-04T17:42:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.700+0000 7f43d27fc640 20 reqs_thread_entry: start 2024-03-04T17:42:12.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.756+0000 7f4469fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:42:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.764+0000 7f45ba1d6980 20 remove_watcher() i=0 2024-03-04T17:42:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.764+0000 7f45ba1d6980 2 removed watcher, disabling cache 2024-03-04T17:42:12.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.768+0000 7f45ba1d6980 20 remove_watcher() i=1 2024-03-04T17:42:12.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.768+0000 7f45ba1d6980 20 remove_watcher() i=2 2024-03-04T17:42:12.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.772+0000 7f45ba1d6980 20 remove_watcher() i=3 2024-03-04T17:42:12.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.772+0000 7f45ba1d6980 20 remove_watcher() i=4 2024-03-04T17:42:12.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.776+0000 7f45ba1d6980 20 remove_watcher() i=5 2024-03-04T17:42:12.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.780+0000 7f45ba1d6980 20 remove_watcher() i=6 2024-03-04T17:42:12.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.780+0000 7f45ba1d6980 20 remove_watcher() i=7 2024-03-04T17:42:12.804 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-04T17:42:12.804 DEBUG:tasks.util.rgw: json result: [] 2024-03-04T17:42:12.805 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-03-04T17:42:12.805 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-03-04T17:42:12.805 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-03-04T17:42:12.847 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:42:12.847 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:42:12.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.872+0000 7faf70f72980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:42:12.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.872+0000 7faf70f72980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:42:12.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:12.876+0000 7fae2cff9640 20 reqs_thread_entry: start 2024-03-04T17:42:12.904 INFO:teuthology.orchestra.run.smithi047.stdout:{"id":"797b67a4-b876-4d04-b122-10afd858d465","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-04T17:42:12.904 DEBUG:tasks.util.rgw: json result: {'id': '797b67a4-b876-4d04-b122-10afd858d465', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-04T17:42:12.904 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-03-04T17:42:12.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-03-04T17:42:12.904 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-03-04T17:42:12.974 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:42:12.974 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:42:13.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.008+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:13.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 20 realm 2024-03-04T17:42:13.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:13.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:42:13.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:13.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:42:13.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.012+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.016+0000 7f4ff30e7980 20 rados_obj.operate() r=0 bl.length=891 2024-03-04T17:42:13.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.016+0000 7f4ff30e7980 20 searching for the correct realm 2024-03-04T17:42:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.024+0000 7f4ff0bff640 0 -- 172.21.15.47:0/2067149543 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f4fe0068430 msgr2=0x7f4fe006c1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:42:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.028+0000 7f4ff0bff640 0 -- 172.21.15.47:0/2067149543 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x5636b81ceb80 msgr2=0x5636b81d0fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:42:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:42:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:42:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:42:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:42:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:42:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:42:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:42:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 rados_obj.operate() r=0 bl.length=398 2024-03-04T17:42:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 zone default found 2024-03-04T17:42:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 4 Realm: () 2024-03-04T17:42:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:42:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:42:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:42:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 zonegroup default 2024-03-04T17:42:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.036+0000 7f4ff30e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.040+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.040+0000 7f4ff30e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.040+0000 7f4ff30e7980 20 rados->read ofs=0 len=0 2024-03-04T17:42:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.040+0000 7f4ff30e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:42:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.040+0000 7f4ff30e7980 20 started sync module instance, tier type = 2024-03-04T17:42:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.040+0000 7f4ff30e7980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:42:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.060+0000 7f4ff30e7980 20 add_watcher() i=0 2024-03-04T17:42:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.060+0000 7f4ff30e7980 20 add_watcher() i=1 2024-03-04T17:42:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.060+0000 7f4ff30e7980 20 add_watcher() i=2 2024-03-04T17:42:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.060+0000 7f4ff30e7980 20 add_watcher() i=3 2024-03-04T17:42:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.060+0000 7f4ff30e7980 20 add_watcher() i=4 2024-03-04T17:42:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.060+0000 7f4ff30e7980 20 add_watcher() i=5 2024-03-04T17:42:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.060+0000 7f4ff30e7980 20 add_watcher() i=6 2024-03-04T17:42:13.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.064+0000 7f4ff30e7980 20 add_watcher() i=7 2024-03-04T17:42:13.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.064+0000 7f4ff30e7980 2 all 8 watchers are set, enabling cache 2024-03-04T17:42:13.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.068+0000 7f4ea6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:42:13.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.068+0000 7f4ff30e7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:42:13.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.068+0000 7f4ff30e7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:42:13.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.068+0000 7f4ff30e7980 5 note: GC not initialized 2024-03-04T17:42:13.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.068+0000 7f4ea5fe3640 20 reqs_thread_entry: start 2024-03-04T17:42:13.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.068+0000 7f4ff30e7980 20 init_complete bucket index max shards: 11 2024-03-04T17:42:13.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.068+0000 7f4ff30e7980 20 Filter name: none 2024-03-04T17:42:13.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.068+0000 7f4e177fe640 20 reqs_thread_entry: start 2024-03-04T17:42:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.088+0000 7f4ea6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:42:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.092+0000 7f4ff30e7980 20 remove_watcher() i=0 2024-03-04T17:42:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.092+0000 7f4ff30e7980 2 removed watcher, disabling cache 2024-03-04T17:42:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.096+0000 7f4ff30e7980 20 remove_watcher() i=1 2024-03-04T17:42:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.096+0000 7f4ff30e7980 20 remove_watcher() i=2 2024-03-04T17:42:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.100+0000 7f4ff30e7980 20 remove_watcher() i=3 2024-03-04T17:42:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.100+0000 7f4ff30e7980 20 remove_watcher() i=4 2024-03-04T17:42:13.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.104+0000 7f4ff30e7980 20 remove_watcher() i=5 2024-03-04T17:42:13.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.104+0000 7f4ff30e7980 20 remove_watcher() i=6 2024-03-04T17:42:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.108+0000 7f4ff30e7980 20 remove_watcher() i=7 2024-03-04T17:42:13.128 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-04T17:42:13.128 DEBUG:tasks.util.rgw: json result: [] 2024-03-04T17:42:13.128 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2024-03-04T17:42:13.128 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-03-04T17:42:13.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-03-04T17:42:13.129 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2024-03-04T17:42:13.213 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:42:13.213 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:42:13.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.240+0000 7f9d842af980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:42:13.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.240+0000 7f9d842af980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:42:13.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.244+0000 7f9bb7fff640 20 reqs_thread_entry: start 2024-03-04T17:42:13.267 INFO:teuthology.orchestra.run.smithi047.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2024-03-04T17:42:13.267 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['LUKEWARM', 'STANDARD']}}] 2024-03-04T17:42:13.267 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-03-04T17:42:13.267 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-03-04T17:42:13.268 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2024-03-04T17:42:13.347 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:42:13.348 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:42:13.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.372+0000 7f5b12448980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:42:13.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.372+0000 7f5b12448980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:42:13.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.376+0000 7f5985ffb640 20 reqs_thread_entry: start 2024-03-04T17:42:13.401 INFO:teuthology.orchestra.run.smithi047.stdout:{"id":"797b67a4-b876-4d04-b122-10afd858d465","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-04T17:42:13.401 DEBUG:tasks.util.rgw: json result: {'id': '797b67a4-b876-4d04-b122-10afd858d465', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-04T17:42:13.401 INFO:tasks.rgw:Configuring storage class type = FROZEN 2024-03-04T17:42:13.401 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-03-04T17:42:13.401 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-03-04T17:42:13.402 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2024-03-04T17:42:13.476 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:42:13.476 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:42:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.500+0000 7fda56a4f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:42:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.500+0000 7fda56a4f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:42:13.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.504+0000 7fd8aa7fc640 20 reqs_thread_entry: start 2024-03-04T17:42:13.528 INFO:teuthology.orchestra.run.smithi047.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2024-03-04T17:42:13.528 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2024-03-04T17:42:13.528 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-03-04T17:42:13.528 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-03-04T17:42:13.528 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2024-03-04T17:42:13.613 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:42:13.613 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:42:13.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.640+0000 7fd8bbf99980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:42:13.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.640+0000 7fd8bbf99980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:42:13.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:42:13.644+0000 7fd77cfe1640 20 reqs_thread_entry: start 2024-03-04T17:42:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:{"id":"797b67a4-b876-4d04-b122-10afd858d465","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-04T17:42:13.669 DEBUG:tasks.util.rgw: json result: {'id': '797b67a4-b876-4d04-b122-10afd858d465', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-04T17:42:13.669 INFO:tasks.rgw:Starting rgw... 2024-03-04T17:42:13.669 INFO:tasks.rgw:rgw client.0 config is {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2024-03-04T17:42:13.669 INFO:tasks.rgw:Using beast as radosgw frontend 2024-03-04T17:42:13.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo echo -n https://smithi047.front.sepia.ceph.com:443 | sudo tee /home/ubuntu/cephtest/url_file 2024-03-04T17:42:13.722 INFO:teuthology.orchestra.run.smithi047.stdout:https://smithi047.front.sepia.ceph.com:443 2024-03-04T17:42:13.722 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-03-04T17:42:13.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo keytool -import -alias smithi047.front.sepia.ceph.com -keystore $(readlink -e $(dirname $(readlink -e $(which keytool)))/../lib/security/cacerts) -file /home/ubuntu/cephtest/ca/rgw.client.0.crt -storepass changeit 2024-03-04T17:42:14.121 INFO:teuthology.orchestra.run.smithi047.stderr:Warning: use -cacerts option to access cacerts keystore 2024-03-04T17:42:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:Certificate was added to keystore 2024-03-04T17:42:14.505 DEBUG:tasks.ceph_manager:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2024-03-04T17:42:14.505 INFO:tasks.rgw.client.0:Restarting daemon 2024-03-04T17:42:14.505 DEBUG:teuthology.orchestra.run.smithi047:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast ssl_certificate=/home/ubuntu/cephtest/ca/rgw.client.0.crt ssl_port=443' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-03-04T17:42:14.508 INFO:tasks.rgw.client.0:Started 2024-03-04T17:42:14.508 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on https://smithi047.front.sepia.ceph.com:443/ 2024-03-04T17:42:14.508 DEBUG:teuthology.orchestra.run.smithi047:> curl https://smithi047.front.sepia.ceph.com:443/ 2024-03-04T17:42:14.528 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:42:14.528 INFO:teuthology.orchestra.run.smithi047.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:42:14.528 INFO:teuthology.orchestra.run.smithi047.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:42:14.528 INFO:teuthology.orchestra.run.smithi047.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:42:14.528 INFO:teuthology.orchestra.run.smithi047.stderr:curl: (7) Failed to connect to smithi047.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:42:15.529 DEBUG:teuthology.orchestra.run.smithi047:> curl https://smithi047.front.sepia.ceph.com:443/ 2024-03-04T17:42:15.541 INFO:teuthology.orchestra.run.smithi047.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:42:15.541 INFO:teuthology.orchestra.run.smithi047.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:42:15.541 INFO:teuthology.orchestra.run.smithi047.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:42:15.541 INFO:teuthology.orchestra.run.smithi047.stderr:curl: (7) Failed to connect to smithi047.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:42:15.542 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:42:17.544 DEBUG:teuthology.orchestra.run.smithi047:> curl https://smithi047.front.sepia.ceph.com:443/ 2024-03-04T17:42:17.556 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:42:17.556 INFO:teuthology.orchestra.run.smithi047.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:42:17.556 INFO:teuthology.orchestra.run.smithi047.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:42:17.556 INFO:teuthology.orchestra.run.smithi047.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:42:17.557 INFO:teuthology.orchestra.run.smithi047.stderr:curl: (7) Failed to connect to smithi047.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:42:21.557 DEBUG:teuthology.orchestra.run.smithi047:> curl https://smithi047.front.sepia.ceph.com:443/ 2024-03-04T17:42:21.569 INFO:teuthology.orchestra.run.smithi047.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:42:21.570 INFO:teuthology.orchestra.run.smithi047.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:42:21.570 INFO:teuthology.orchestra.run.smithi047.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:42:21.570 INFO:teuthology.orchestra.run.smithi047.stderr:curl: (7) Failed to connect to smithi047.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:42:21.570 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:42:29.572 DEBUG:teuthology.orchestra.run.smithi047:> curl https://smithi047.front.sepia.ceph.com:443/ 2024-03-04T17:42:29.584 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:42:29.585 INFO:teuthology.orchestra.run.smithi047.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:42:29.585 INFO:teuthology.orchestra.run.smithi047.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:42:29.585 INFO:teuthology.orchestra.run.smithi047.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:42:29.585 INFO:teuthology.orchestra.run.smithi047.stderr:curl: (7) Failed to connect to smithi047.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:42:40.194 INFO:tasks.rgw.client.0.smithi047.stdout:2024-03-04T17:42:40.187+0000 8e90e80 -1 ssl_private_key was not found: rgw/cert//default.key 2024-03-04T17:42:45.587 DEBUG:teuthology.orchestra.run.smithi047:> curl https://smithi047.front.sepia.ceph.com:443/ 2024-03-04T17:42:45.599 INFO:teuthology.orchestra.run.smithi047.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:42:45.599 INFO:teuthology.orchestra.run.smithi047.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:42:46.821 INFO:teuthology.orchestra.run.smithi047.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 175 0 --:--:-- 0:00:01 --:--:-- 175 2024-03-04T17:42:46.825 INFO:teuthology.orchestra.run.smithi047.stdout:anonymous 2024-03-04T17:42:46.826 INFO:teuthology.run_tasks:Running task tox... 2024-03-04T17:42:46.837 INFO:tasks.tox:Deploying tox from pip... 2024-03-04T17:42:46.837 DEBUG:teuthology.orchestra.run.smithi047:> python3 -m venv /home/ubuntu/cephtest/tox-venv 2024-03-04T17:42:49.889 DEBUG:teuthology.orchestra.run.smithi047:> source /home/ubuntu/cephtest/tox-venv/bin/activate && pip install tox==3.15.0 2024-03-04T17:42:50.749 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting tox==3.15.0 2024-03-04T17:42:50.965 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading tox-3.15.0-py2.py3-none-any.whl (82 kB) 2024-03-04T17:42:51.074 INFO:teuthology.orchestra.run.smithi047.stdout: --------------------------------------- 82.1/82.1 KB 743.6 kB/s eta 0:00:00 2024-03-04T17:42:51.259 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting pluggy<1,>=0.12.0 2024-03-04T17:42:51.310 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) 2024-03-04T17:42:51.646 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 2024-03-04T17:42:51.699 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading virtualenv-20.25.1-py3-none-any.whl (3.8 MB) 2024-03-04T17:42:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: ---------------------------------------- 3.8/3.8 MB 4.1 MB/s eta 0:00:00 2024-03-04T17:42:52.808 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting filelock<4,>=3.0.0 2024-03-04T17:42:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading filelock-3.13.1-py3-none-any.whl (11 kB) 2024-03-04T17:42:53.041 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting py<2,>=1.4.17 2024-03-04T17:42:53.098 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading py-1.11.0-py2.py3-none-any.whl (98 kB) 2024-03-04T17:42:53.144 INFO:teuthology.orchestra.run.smithi047.stdout: ---------------------------------------- 98.7/98.7 KB 2.2 MB/s eta 0:00:00 2024-03-04T17:42:53.255 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting toml>=0.9.4 2024-03-04T17:42:53.315 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-03-04T17:42:53.511 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting packaging>=14 2024-03-04T17:42:53.567 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading packaging-23.2-py3-none-any.whl (53 kB) 2024-03-04T17:42:53.604 INFO:teuthology.orchestra.run.smithi047.stdout: ---------------------------------------- 53.0/53.0 KB 1.3 MB/s eta 0:00:00 2024-03-04T17:42:53.721 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting six<2,>=1.14.0 2024-03-04T17:42:53.774 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-03-04T17:42:54.011 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting distlib<1,>=0.3.7 2024-03-04T17:42:54.066 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB) 2024-03-04T17:42:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: --------------------------------------- 468.9/468.9 KB 4.5 MB/s eta 0:00:00 2024-03-04T17:42:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:Collecting platformdirs<5,>=3.9.1 2024-03-04T17:42:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: Downloading platformdirs-4.2.0-py3-none-any.whl (17 kB) 2024-03-04T17:42:54.599 INFO:teuthology.orchestra.run.smithi047.stdout:Installing collected packages: distlib, toml, six, py, pluggy, platformdirs, packaging, filelock, virtualenv, tox 2024-03-04T17:42:55.565 INFO:teuthology.orchestra.run.smithi047.stdout:Successfully installed distlib-0.3.8 filelock-3.13.1 packaging-23.2 platformdirs-4.2.0 pluggy-0.13.1 py-1.11.0 six-1.16.0 toml-0.10.2 tox-3.15.0 virtualenv-20.25.1 2024-03-04T17:42:55.657 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T17:42:55.670 INFO:tasks.workunit:Pulling workunits from ref 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:42:55.671 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T17:42:55.671 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:42:55.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:42:55.678 INFO:teuthology.orchestra.run.smithi047.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T17:42:55.678 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:42:55.727 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T17:42:55.727 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T17:42:55.775 INFO:tasks.workunit:timeout=3h 2024-03-04T17:42:55.775 INFO:tasks.workunit:cleanup=True 2024-03-04T17:42:55.775 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/pritha-srivastava/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:42:55.825 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T17:44:54.680 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to '3aec390336fdfdc8f927818b718cfb0361bd47c8'. 2024-03-04T17:44:54.681 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:44:54.681 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T17:44:54.681 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T17:44:54.681 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T17:44:54.681 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:44:54.681 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T17:44:54.681 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T17:44:54.681 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:44:54.682 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-03-04T17:44:54.682 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:44:54.682 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-03-04T17:44:54.682 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:44:54.682 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-03-04T17:44:54.682 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:44:54.682 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T17:44:54.682 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:44:54.682 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at 3aec390336f qa/d4n: Start redis up before rgw 2024-03-04T17:44:54.686 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T17:44:54.736 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T17:44:54.739 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:44:54.739 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T17:44:55.515 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T17:44:55.594 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T17:44:55.640 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:44:55.642 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:44:55.642 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T17:44:55.684 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:44:55.688 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:44:55.688 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T17:44:55.734 INFO:tasks.workunit:Running workunits matching rgw/run-bucket-check.sh on client.0... 2024-03-04T17:44:55.736 INFO:tasks.workunit:Running workunit rgw/run-bucket-check.sh... 2024-03-04T17:44:55.736 DEBUG:teuthology.orchestra.run.smithi047:workunit test rgw/run-bucket-check.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-03-04T17:44:55.786 INFO:tasks.workunit.client.0.smithi047.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-03-04T17:44:55.787 INFO:tasks.workunit.client.0.smithi047.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T17:44:55.787 INFO:tasks.workunit.client.0.smithi047.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ deactivate nondestructive 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n '' ']' 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n '' ']' 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ hash -r 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n '' ']' 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ unset VIRTUAL_ENV 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-03-04T17:44:58.772 INFO:tasks.workunit.client.0.smithi047.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T17:44:58.773 INFO:tasks.workunit.client.0.smithi047.stderr:++ export VIRTUAL_ENV 2024-03-04T17:44:58.773 INFO:tasks.workunit.client.0.smithi047.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T17:44:58.773 INFO:tasks.workunit.client.0.smithi047.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ export PATH 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n '' ']' 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -z '' ']' 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ _OLD_VIRTUAL_PS1= 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ PS1='(rgw) ' 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ export PS1 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ export VIRTUAL_ENV_PROMPT 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:++ hash -r 2024-03-04T17:44:58.774 INFO:tasks.workunit.client.0.smithi047.stderr:+ pip install pip --upgrade 2024-03-04T17:44:59.144 INFO:tasks.workunit.client.0.smithi047.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-03-04T17:44:59.672 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting pip 2024-03-04T17:44:59.921 INFO:tasks.workunit.client.0.smithi047.stdout: Downloading pip-24.0-py3-none-any.whl (2.1 MB) 2024-03-04T17:45:00.290 INFO:tasks.workunit.client.0.smithi047.stdout: ---------------------------------------- 2.1/2.1 MB 5.9 MB/s eta 0:00:00 2024-03-04T17:45:00.365 INFO:tasks.workunit.client.0.smithi047.stdout:Installing collected packages: pip 2024-03-04T17:45:00.365 INFO:tasks.workunit.client.0.smithi047.stdout: Attempting uninstall: pip 2024-03-04T17:45:00.366 INFO:tasks.workunit.client.0.smithi047.stdout: Found existing installation: pip 22.0.2 2024-03-04T17:45:00.664 INFO:tasks.workunit.client.0.smithi047.stdout: Uninstalling pip-22.0.2: 2024-03-04T17:45:00.670 INFO:tasks.workunit.client.0.smithi047.stdout: Successfully uninstalled pip-22.0.2 2024-03-04T17:45:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:Successfully installed pip-24.0 2024-03-04T17:45:01.737 INFO:tasks.workunit.client.0.smithi047.stderr:+ pip install boto3 2024-03-04T17:45:02.985 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting boto3 2024-03-04T17:45:03.119 INFO:tasks.workunit.client.0.smithi047.stdout: Downloading boto3-1.34.54-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T17:45:03.885 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting botocore<1.35.0,>=1.34.54 (from boto3) 2024-03-04T17:45:03.921 INFO:tasks.workunit.client.0.smithi047.stdout: Downloading botocore-1.34.54-py3-none-any.whl.metadata (5.7 kB) 2024-03-04T17:45:04.100 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-03-04T17:45:04.143 INFO:tasks.workunit.client.0.smithi047.stdout: Downloading jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-03-04T17:45:04.320 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-03-04T17:45:04.361 INFO:tasks.workunit.client.0.smithi047.stdout: Downloading s3transfer-0.10.0-py3-none-any.whl.metadata (1.7 kB) 2024-03-04T17:45:04.535 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:45:04.579 INFO:tasks.workunit.client.0.smithi047.stdout: Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-03-04T17:45:04.827 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting urllib3<2.1,>=1.25.4 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:45:04.859 INFO:tasks.workunit.client.0.smithi047.stdout: Downloading urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T17:45:05.227 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:45:05.259 INFO:tasks.workunit.client.0.smithi047.stdout: Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-03-04T17:45:05.375 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading boto3-1.34.54-py3-none-any.whl (139 kB) 2024-03-04T17:45:05.508 INFO:tasks.workunit.client.0.smithi047.stdout: ---------------------------------------- 139.3/139.3 kB 1.0 MB/s eta 0:00:00 2024-03-04T17:45:05.551 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading botocore-1.34.54-py3-none-any.whl (12.0 MB) 2024-03-04T17:45:07.621 INFO:tasks.workunit.client.0.smithi047.stdout: ---------------------------------------- 12.0/12.0 MB 7.4 MB/s eta 0:00:00 2024-03-04T17:45:07.657 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-03-04T17:45:07.745 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading s3transfer-0.10.0-py3-none-any.whl (82 kB) 2024-03-04T17:45:07.801 INFO:tasks.workunit.client.0.smithi047.stdout: ---------------------------------------- 82.1/82.1 kB 1.4 MB/s eta 0:00:00 2024-03-04T17:45:07.837 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-03-04T17:45:07.901 INFO:tasks.workunit.client.0.smithi047.stdout: ---------------------------------------- 229.9/229.9 kB 3.7 MB/s eta 0:00:00 2024-03-04T17:45:07.935 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading urllib3-2.0.7-py3-none-any.whl (124 kB) 2024-03-04T17:45:07.985 INFO:tasks.workunit.client.0.smithi047.stdout: ---------------------------------------- 124.2/124.2 kB 2.5 MB/s eta 0:00:00 2024-03-04T17:45:08.018 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-03-04T17:45:08.207 INFO:tasks.workunit.client.0.smithi047.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-03-04T17:45:09.405 INFO:tasks.workunit.client.0.smithi047.stdout:Successfully installed boto3-1.34.54 botocore-1.34.54 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.0 six-1.16.0 urllib3-2.0.7 2024-03-04T17:45:10.016 INFO:tasks.workunit.client.0.smithi047.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_bucket_check.py 2024-03-04T17:45:10.164 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin user create --uid check-tester --display-name "Check Testing" --access-key OJODXSLNX4LUNHQG99PA --secret 3l6ffld34qaymfomuh832j94738aie2x4p2o8h6n 2024-03-04T17:45:10.202 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:10.202 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:10.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.244+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 20 realm 2024-03-04T17:45:10.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:10.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:10.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.248+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.252+0000 7f7635aa0980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:10.254 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.252+0000 7f7635aa0980 20 searching for the correct realm 2024-03-04T17:45:10.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.256+0000 7f76335b8640 0 -- 172.21.15.47:0/3943832115 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f76240a6c50 msgr2=0x7f7624115af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:10.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.468+0000 7f76335b8640 0 -- 172.21.15.47:0/3943832115 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f761c003ac0 msgr2=0x7f762c0403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:10.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:10.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:10.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:10.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:10.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:10.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:10.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:10.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:10.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 zone default found 2024-03-04T17:45:10.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 4 Realm: () 2024-03-04T17:45:10.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:10.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:10.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:10.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 zonegroup default 2024-03-04T17:45:10.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.688+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.692+0000 7f7635aa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.692+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.692+0000 7f7635aa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.692+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.692+0000 7f7635aa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.692+0000 7f7635aa0980 20 started sync module instance, tier type = 2024-03-04T17:45:10.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.692+0000 7f7635aa0980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:10.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.716+0000 7f7635aa0980 20 add_watcher() i=0 2024-03-04T17:45:10.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.716+0000 7f7635aa0980 20 add_watcher() i=1 2024-03-04T17:45:10.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.716+0000 7f7635aa0980 20 add_watcher() i=2 2024-03-04T17:45:10.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.716+0000 7f7635aa0980 20 add_watcher() i=3 2024-03-04T17:45:10.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.716+0000 7f7635aa0980 20 add_watcher() i=4 2024-03-04T17:45:10.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.716+0000 7f7635aa0980 20 add_watcher() i=5 2024-03-04T17:45:10.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.716+0000 7f7635aa0980 20 add_watcher() i=6 2024-03-04T17:45:10.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.720+0000 7f7635aa0980 20 add_watcher() i=7 2024-03-04T17:45:10.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.720+0000 7f7635aa0980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:10.729 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.724+0000 7f74e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:10.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:10.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:10.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 5 note: GC not initialized 2024-03-04T17:45:10.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f74e8fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 20 Filter name: none 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f74627fc640 20 reqs_thread_entry: start 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 10 cache get: name=default.rgw.meta+users.uid+check-tester : miss 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x0 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 10 adding default.rgw.meta+users.uid+check-tester to cache LRU end 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : miss 2024-03-04T17:45:10.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x0 2024-03-04T17:45:10.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 10 adding default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-03-04T17:45:10.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-03-04T17:45:10.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.728+0000 7f7635aa0980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-03-04T17:45:10.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.732+0000 7f7635aa0980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-03-04T17:45:10.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.732+0000 7f7635aa0980 10 moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-03-04T17:45:10.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.732+0000 7f7635aa0980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:check-tester, ofs0, ns] 2024-03-04T17:45:10.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.732+0000 7f7516ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 154618822656 cookie 94381128157888 notifier 4505 bl.length()=560 2024-03-04T17:45:10.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.732+0000 7f7516ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-03-04T17:45:10.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.732+0000 7f7516ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-03-04T17:45:10.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.776+0000 7f7635aa0980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-03-04T17:45:10.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.776+0000 7f7635aa0980 10 moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-03-04T17:45:10.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.776+0000 7f7635aa0980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:OJODXSLNX4LUNHQG99PA, ofs0, ns] 2024-03-04T17:45:10.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.776+0000 7f7516ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 154618822656 cookie 94381128151824 notifier 4505 bl.length()=186 2024-03-04T17:45:10.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.776+0000 7f7516ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-03-04T17:45:10.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.776+0000 7f7516ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-03-04T17:45:10.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.780+0000 7f74e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:10.789 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.784+0000 7f7635aa0980 20 remove_watcher() i=0 2024-03-04T17:45:10.789 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.784+0000 7f7635aa0980 2 removed watcher, disabling cache 2024-03-04T17:45:10.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.788+0000 7f7635aa0980 20 remove_watcher() i=1 2024-03-04T17:45:10.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.792+0000 7f7635aa0980 20 remove_watcher() i=2 2024-03-04T17:45:10.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.792+0000 7f7635aa0980 20 remove_watcher() i=3 2024-03-04T17:45:10.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.796+0000 7f7635aa0980 20 remove_watcher() i=4 2024-03-04T17:45:10.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.796+0000 7f7635aa0980 20 remove_watcher() i=5 2024-03-04T17:45:10.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.800+0000 7f7635aa0980 20 remove_watcher() i=6 2024-03-04T17:45:10.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:10.804+0000 7f7635aa0980 20 remove_watcher() i=7 2024-03-04T17:45:11.075 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.075 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:11.336 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.336 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:11.516 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.516 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:12.159 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:12.160 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:12.852 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:12.852 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:13.123 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:13.123 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:13.212 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:13.212 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:13.277 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:13.277 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:13.465 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:13.465 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:13.858 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:13.859 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:13.952 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:13.952 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:14.090 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:14.090 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:14.156 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:14.156 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:14.288 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:14.288 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:14.352 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:14.352 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:14.564 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:14.564 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:14.627 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: recalculated bucket check stats are correct 2024-03-04T17:45:14.627 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:14.627 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-03-04T17:45:14.663 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:14.664 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:14.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.704+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 20 realm 2024-03-04T17:45:14.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:14.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:14.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.708+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.712+0000 7fabbf550980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:14.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.712+0000 7fabbf550980 20 searching for the correct realm 2024-03-04T17:45:14.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.716+0000 7fabbd869640 0 -- 172.21.15.47:0/2634749726 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7faba803d070 msgr2=0x7faba8057960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:14.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.724+0000 7fabbd068640 0 -- 172.21.15.47:0/2634749726 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fabb810a130 msgr2=0x7fabb81220b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:14.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:14.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:14.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:14.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:14.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:14.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:14.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:14.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:14.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 zone default found 2024-03-04T17:45:14.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 4 Realm: () 2024-03-04T17:45:14.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:14.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:14.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:14.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 zonegroup default 2024-03-04T17:45:14.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.732+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.736+0000 7fabbf550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.736+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.736+0000 7fabbf550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.736+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.736+0000 7fabbf550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.736+0000 7fabbf550980 20 started sync module instance, tier type = 2024-03-04T17:45:14.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.736+0000 7fabbf550980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:14.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.756+0000 7fabbf550980 20 add_watcher() i=0 2024-03-04T17:45:14.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.756+0000 7fabbf550980 20 add_watcher() i=1 2024-03-04T17:45:14.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.756+0000 7fabbf550980 20 add_watcher() i=2 2024-03-04T17:45:14.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.756+0000 7fabbf550980 20 add_watcher() i=3 2024-03-04T17:45:14.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.756+0000 7fabbf550980 20 add_watcher() i=4 2024-03-04T17:45:14.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.756+0000 7fabbf550980 20 add_watcher() i=5 2024-03-04T17:45:14.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.756+0000 7fabbf550980 20 add_watcher() i=6 2024-03-04T17:45:14.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.756+0000 7fabbf550980 20 add_watcher() i=7 2024-03-04T17:45:14.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.756+0000 7fabbf550980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:14.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.760+0000 7faa737e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:14.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.760+0000 7fabbf550980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:14.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.760+0000 7fabbf550980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:14.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.760+0000 7fabbf550980 5 note: GC not initialized 2024-03-04T17:45:14.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.760+0000 7faa727e4640 20 reqs_thread_entry: start 2024-03-04T17:45:14.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.760+0000 7fabbf550980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:14.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 20 Filter name: none 2024-03-04T17:45:14.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fa9e3fff640 20 reqs_thread_entry: start 2024-03-04T17:45:14.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:14.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:14.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:14.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:14.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:14.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:14.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:14.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:14.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:14.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:14.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:14.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 20 list_objects_ordered: starting attempt 1 2024-03-04T17:45:14.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T17:45:14.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T17:45:14.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T17:45:14.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T17:45:14.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.764+0000 7fabbf550980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T17:45:14.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.772+0000 7faa737e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:14.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.776+0000 7fabbf550980 20 remove_watcher() i=0 2024-03-04T17:45:14.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.776+0000 7fabbf550980 2 removed watcher, disabling cache 2024-03-04T17:45:14.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.780+0000 7fabbf550980 20 remove_watcher() i=1 2024-03-04T17:45:14.786 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.784+0000 7fabbf550980 20 remove_watcher() i=2 2024-03-04T17:45:14.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.784+0000 7fabbf550980 20 remove_watcher() i=3 2024-03-04T17:45:14.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.788+0000 7fabbf550980 20 remove_watcher() i=4 2024-03-04T17:45:14.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.788+0000 7fabbf550980 20 remove_watcher() i=5 2024-03-04T17:45:14.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.792+0000 7fabbf550980 20 remove_watcher() i=6 2024-03-04T17:45:14.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.796+0000 7fabbf550980 20 remove_watcher() i=7 2024-03-04T17:45:14.812 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-03-04T17:45:14.839 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:14.839 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:14.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.880+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 realm 2024-03-04T17:45:14.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:14.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:14.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:14.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.884+0000 7f0e2e4e0980 20 searching for the correct realm 2024-03-04T17:45:14.896 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.892+0000 7f0e2b7f7640 0 -- 172.21.15.47:0/2976603153 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f0e200d3020 msgr2=0x7f0e20120090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:14.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.900+0000 7f0e2c7f9640 0 -- 172.21.15.47:0/2976603153 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55c6ce792ea0 msgr2=0x55c6ce7952e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:14.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:14.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:14.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:14.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:14.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:14.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:14.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 zone default found 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 4 Realm: () 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 zonegroup default 2024-03-04T17:45:14.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 started sync module instance, tier type = 2024-03-04T17:45:14.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.908+0000 7f0e2e4e0980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:14.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.928+0000 7f0e2e4e0980 20 add_watcher() i=0 2024-03-04T17:45:14.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.928+0000 7f0e2e4e0980 20 add_watcher() i=1 2024-03-04T17:45:14.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.928+0000 7f0e2e4e0980 20 add_watcher() i=2 2024-03-04T17:45:14.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.928+0000 7f0e2e4e0980 20 add_watcher() i=3 2024-03-04T17:45:14.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.928+0000 7f0e2e4e0980 20 add_watcher() i=4 2024-03-04T17:45:14.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.928+0000 7f0e2e4e0980 20 add_watcher() i=5 2024-03-04T17:45:14.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.928+0000 7f0e2e4e0980 20 add_watcher() i=6 2024-03-04T17:45:14.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.932+0000 7f0e2e4e0980 20 add_watcher() i=7 2024-03-04T17:45:14.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.932+0000 7f0e2e4e0980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:14.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.936+0000 7f0ce1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:14.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.936+0000 7f0e2e4e0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:14.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.936+0000 7f0e2e4e0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:14.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.936+0000 7f0e2e4e0980 5 note: GC not initialized 2024-03-04T17:45:14.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.936+0000 7f0ce0fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:14.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.936+0000 7f0e2e4e0980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:14.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.936+0000 7f0e2e4e0980 20 Filter name: none 2024-03-04T17:45:14.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.936+0000 7f0c5a7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:14.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.936+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:14.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:14.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:14.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:14.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:14.944 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:14.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.940+0000 7f0e2e4e0980 20 rados->read ofs=0 len=0 2024-03-04T17:45:14.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.944+0000 7f0e2e4e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:14.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.944+0000 7f0ce1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:14.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.948+0000 7f0e2e4e0980 20 remove_watcher() i=0 2024-03-04T17:45:14.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.948+0000 7f0e2e4e0980 2 removed watcher, disabling cache 2024-03-04T17:45:14.954 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.952+0000 7f0e2e4e0980 20 remove_watcher() i=1 2024-03-04T17:45:14.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.952+0000 7f0e2e4e0980 20 remove_watcher() i=2 2024-03-04T17:45:14.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.956+0000 7f0e2e4e0980 20 remove_watcher() i=3 2024-03-04T17:45:14.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.956+0000 7f0e2e4e0980 20 remove_watcher() i=4 2024-03-04T17:45:14.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.960+0000 7f0e2e4e0980 20 remove_watcher() i=5 2024-03-04T17:45:14.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.960+0000 7f0e2e4e0980 20 remove_watcher() i=6 2024-03-04T17:45:14.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:14.964+0000 7f0e2e4e0980 20 remove_watcher() i=7 2024-03-04T17:45:14.982 INFO:tasks.workunit.client.0.smithi047.stderr:{'rgw.main': {'size': 54, 'size_actual': 24576, 'size_utilized': 89, 'size_kb': 1, 'size_kb_actual': 24, 'size_kb_utilized': 1, 'num_objects': 6}} 2024-03-04T17:45:14.982 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: bucket check unlinked does not report normal entries 2024-03-04T17:45:14.982 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:14.983 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-03-04T17:45:15.014 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:15.014 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:15.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.052+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 20 realm 2024-03-04T17:45:15.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:15.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:15.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.056+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.060+0000 7f4f8c4e9980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:15.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.060+0000 7f4f8c4e9980 20 searching for the correct realm 2024-03-04T17:45:15.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.064+0000 7f4f8a001640 0 -- 172.21.15.47:0/3967856547 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f4f7c03d930 msgr2=0x7f4f7c0313f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:15.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.072+0000 7f4f8a001640 0 -- 172.21.15.47:0/3967856547 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x55d9905a1b50 msgr2=0x55d9905a3f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:15.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:15.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:15.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:15.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:15.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:15.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:15.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:15.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 zone default found 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 4 Realm: () 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 zonegroup default 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.080+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.084+0000 7f4f8c4e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.084+0000 7f4f8c4e9980 20 started sync module instance, tier type = 2024-03-04T17:45:15.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.084+0000 7f4f8c4e9980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:15.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.104+0000 7f4f8c4e9980 20 add_watcher() i=0 2024-03-04T17:45:15.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.104+0000 7f4f8c4e9980 20 add_watcher() i=1 2024-03-04T17:45:15.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.104+0000 7f4f8c4e9980 20 add_watcher() i=2 2024-03-04T17:45:15.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.104+0000 7f4f8c4e9980 20 add_watcher() i=3 2024-03-04T17:45:15.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.104+0000 7f4f8c4e9980 20 add_watcher() i=4 2024-03-04T17:45:15.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.104+0000 7f4f8c4e9980 20 add_watcher() i=5 2024-03-04T17:45:15.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.104+0000 7f4f8c4e9980 20 add_watcher() i=6 2024-03-04T17:45:15.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.108+0000 7f4f8c4e9980 20 add_watcher() i=7 2024-03-04T17:45:15.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.108+0000 7f4f8c4e9980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:15.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4e3b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:15.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:15.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:15.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 5 note: GC not initialized 2024-03-04T17:45:15.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4e3a7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:15.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:15.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 20 Filter name: none 2024-03-04T17:45:15.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4dabfff640 20 reqs_thread_entry: start 2024-03-04T17:45:15.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:15.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:15.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:15.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:15.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:15.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:15.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:15.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:15.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:15.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:15.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:15.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.112+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:15.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 1 NOTICE: finished shard 0 (0 entries found) 2024-03-04T17:45:15.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:15.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 1 NOTICE: finished shard 1 (0 entries found) 2024-03-04T17:45:15.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:15.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 1 NOTICE: finished shard 2 (0 entries found) 2024-03-04T17:45:15.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:15.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 1 NOTICE: finished shard 3 (0 entries found) 2024-03-04T17:45:15.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:15.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 1 NOTICE: finished shard 4 (0 entries found) 2024-03-04T17:45:15.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.116+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:15.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 1 NOTICE: finished shard 5 (0 entries found) 2024-03-04T17:45:15.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:15.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 1 NOTICE: finished shard 6 (0 entries found) 2024-03-04T17:45:15.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:15.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 1 NOTICE: finished shard 7 (0 entries found) 2024-03-04T17:45:15.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:15.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 1 NOTICE: finished shard 8 (0 entries found) 2024-03-04T17:45:15.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:15.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 1 NOTICE: finished shard 9 (0 entries found) 2024-03-04T17:45:15.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.120+0000 7f4f8c4e9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:15.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.124+0000 7f4f8c4e9980 1 NOTICE: finished shard 10 (0 entries found) 2024-03-04T17:45:15.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.124+0000 7f4f8c4e9980 1 NOTICE: finished all shards (0 entries found) 2024-03-04T17:45:15.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.124+0000 7f4e3b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:15.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.128+0000 7f4f8c4e9980 20 remove_watcher() i=0 2024-03-04T17:45:15.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.128+0000 7f4f8c4e9980 2 removed watcher, disabling cache 2024-03-04T17:45:15.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.132+0000 7f4f8c4e9980 20 remove_watcher() i=1 2024-03-04T17:45:15.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.132+0000 7f4f8c4e9980 20 remove_watcher() i=2 2024-03-04T17:45:15.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.136+0000 7f4f8c4e9980 20 remove_watcher() i=3 2024-03-04T17:45:15.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.136+0000 7f4f8c4e9980 20 remove_watcher() i=4 2024-03-04T17:45:15.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.140+0000 7f4f8c4e9980 20 remove_watcher() i=5 2024-03-04T17:45:15.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.140+0000 7f4f8c4e9980 20 remove_watcher() i=6 2024-03-04T17:45:15.148 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:15.144+0000 7f4f8c4e9980 20 remove_watcher() i=7 2024-03-04T17:45:15.162 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-03-04T17:45:15.586 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-03-04T17:45:17.006 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:17.006 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:17.175 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:17.175 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-03-04T17:45:17.211 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:17.211 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:17.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.252+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 realm 2024-03-04T17:45:17.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:17.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:17.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:17.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.256+0000 7fd85e38e980 20 searching for the correct realm 2024-03-04T17:45:17.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.264+0000 7fd85c6a7640 0 -- 172.21.15.47:0/2820160543 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7fd844080040 msgr2=0x7fd84405e7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:17.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.272+0000 7fd85bea6640 0 -- 172.21.15.47:0/2820160543 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fd854088040 msgr2=0x7fd854058370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:17.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:17.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:17.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:17.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:17.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:17.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:17.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:17.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:17.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 zone default found 2024-03-04T17:45:17.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 4 Realm: () 2024-03-04T17:45:17.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:17.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:17.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:17.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 zonegroup default 2024-03-04T17:45:17.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 started sync module instance, tier type = 2024-03-04T17:45:17.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.280+0000 7fd85e38e980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:17.308 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.304+0000 7fd85e38e980 20 add_watcher() i=0 2024-03-04T17:45:17.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.304+0000 7fd85e38e980 20 add_watcher() i=1 2024-03-04T17:45:17.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.304+0000 7fd85e38e980 20 add_watcher() i=2 2024-03-04T17:45:17.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.304+0000 7fd85e38e980 20 add_watcher() i=3 2024-03-04T17:45:17.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.304+0000 7fd85e38e980 20 add_watcher() i=4 2024-03-04T17:45:17.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.304+0000 7fd85e38e980 20 add_watcher() i=5 2024-03-04T17:45:17.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.304+0000 7fd85e38e980 20 add_watcher() i=6 2024-03-04T17:45:17.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.308+0000 7fd85e38e980 20 add_watcher() i=7 2024-03-04T17:45:17.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.308+0000 7fd85e38e980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:17.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd711fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:17.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd85e38e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:17.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd85e38e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:17.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd85e38e980 5 note: GC not initialized 2024-03-04T17:45:17.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd710fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:17.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd85e38e980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:17.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd85e38e980 20 Filter name: none 2024-03-04T17:45:17.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd68a7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:17.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd85e38e980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:17.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.312+0000 7fd85e38e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:17.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:17.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:17.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: starting shard=0 2024-03-04T17:45:17.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:17.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: starting shard=1 2024-03-04T17:45:17.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:17.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: starting shard=2 2024-03-04T17:45:17.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:17.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: starting shard=3 2024-03-04T17:45:17.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:17.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 INFO: main: starting shard=4 2024-03-04T17:45:17.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.316+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:17.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 INFO: main: starting shard=5 2024-03-04T17:45:17.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:17.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 INFO: main: starting shard=6 2024-03-04T17:45:17.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:17.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 INFO: main: starting shard=7 2024-03-04T17:45:17.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:17.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 INFO: main: starting shard=8 2024-03-04T17:45:17.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.320+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:17.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.324+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-03-04T17:45:17.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.324+0000 7fd85e38e980 20 INFO: main: starting shard=9 2024-03-04T17:45:17.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.324+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:17.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.324+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.324+0000 7fd85e38e980 20 INFO: main: starting shard=10 2024-03-04T17:45:17.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.324+0000 7fd85e38e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:17.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.324+0000 7fd85e38e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.324+0000 7fd85e38e980 20 INFO: main: done 2024-03-04T17:45:17.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.328+0000 7fd711fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:17.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.332+0000 7fd85e38e980 20 remove_watcher() i=0 2024-03-04T17:45:17.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.332+0000 7fd85e38e980 2 removed watcher, disabling cache 2024-03-04T17:45:17.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.332+0000 7fd85e38e980 20 remove_watcher() i=1 2024-03-04T17:45:17.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.336+0000 7fd85e38e980 20 remove_watcher() i=2 2024-03-04T17:45:17.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.336+0000 7fd85e38e980 20 remove_watcher() i=3 2024-03-04T17:45:17.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.340+0000 7fd85e38e980 20 remove_watcher() i=4 2024-03-04T17:45:17.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.344+0000 7fd85e38e980 20 remove_watcher() i=5 2024-03-04T17:45:17.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.344+0000 7fd85e38e980 20 remove_watcher() i=6 2024-03-04T17:45:17.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.348+0000 7fd85e38e980 20 remove_watcher() i=7 2024-03-04T17:45:17.368 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:17.368 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:17.396 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:17.396 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:17.499 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:17.499 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-03-04T17:45:17.536 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:17.536 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:17.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.580+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.580+0000 7fe639769980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.580+0000 7fe639769980 20 realm 2024-03-04T17:45:17.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.580+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.580+0000 7fe639769980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.580+0000 7fe639769980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:17.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.580+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.580+0000 7fe639769980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.580+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.584+0000 7fe639769980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:17.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.584+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.584+0000 7fe639769980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:17.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.584+0000 7fe639769980 20 searching for the correct realm 2024-03-04T17:45:17.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.588+0000 7fe636a80640 0 -- 172.21.15.47:0/4266732564 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fe62c079630 msgr2=0x7fe62c05d6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:17.805 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.800+0000 7fe636a80640 0 -- 172.21.15.47:0/4266732564 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55855bd8e6c0 msgr2=0x55855bd90af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:17.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.812+0000 7fe639769980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:17.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.812+0000 7fe639769980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:17.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.812+0000 7fe639769980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:17.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.812+0000 7fe639769980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:17.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.812+0000 7fe639769980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:17.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.812+0000 7fe639769980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:17.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.812+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.812+0000 7fe639769980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.812+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:17.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:17.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 zone default found 2024-03-04T17:45:17.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 4 Realm: () 2024-03-04T17:45:17.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:17.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:17.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:17.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 zonegroup default 2024-03-04T17:45:17.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.816+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.820+0000 7fe639769980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.820+0000 7fe639769980 20 started sync module instance, tier type = 2024-03-04T17:45:17.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.820+0000 7fe639769980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:17.848 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.844+0000 7fe639769980 20 add_watcher() i=0 2024-03-04T17:45:17.848 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.844+0000 7fe639769980 20 add_watcher() i=1 2024-03-04T17:45:17.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.844+0000 7fe639769980 20 add_watcher() i=2 2024-03-04T17:45:17.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.844+0000 7fe639769980 20 add_watcher() i=3 2024-03-04T17:45:17.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.844+0000 7fe639769980 20 add_watcher() i=4 2024-03-04T17:45:17.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.844+0000 7fe639769980 20 add_watcher() i=5 2024-03-04T17:45:17.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.844+0000 7fe639769980 20 add_watcher() i=6 2024-03-04T17:45:17.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.848+0000 7fe639769980 20 add_watcher() i=7 2024-03-04T17:45:17.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.848+0000 7fe639769980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:17.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe4edfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:17.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe639769980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:17.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe639769980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:17.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe639769980 5 note: GC not initialized 2024-03-04T17:45:17.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe4ecfe1640 20 reqs_thread_entry: start 2024-03-04T17:45:17.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe639769980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:17.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe639769980 20 Filter name: none 2024-03-04T17:45:17.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe4667fc640 20 reqs_thread_entry: start 2024-03-04T17:45:17.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe639769980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:17.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.856+0000 7fe639769980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:17.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:17.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:17.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 INFO: main: starting shard=0 2024-03-04T17:45:17.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:17.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 INFO: main: starting shard=1 2024-03-04T17:45:17.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:17.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 INFO: main: starting shard=2 2024-03-04T17:45:17.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:17.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 INFO: main: starting shard=3 2024-03-04T17:45:17.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.860+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:17.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.864+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.864+0000 7fe639769980 20 INFO: main: starting shard=4 2024-03-04T17:45:17.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.864+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:17.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.864+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.864+0000 7fe639769980 20 INFO: main: starting shard=5 2024-03-04T17:45:17.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.864+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:17.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.864+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-03-04T17:45:17.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 INFO: main: starting shard=6 2024-03-04T17:45:17.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:17.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 INFO: main: starting shard=7 2024-03-04T17:45:17.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:17.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 INFO: main: starting shard=8 2024-03-04T17:45:17.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:17.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 INFO: main: starting shard=9 2024-03-04T17:45:17.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:17.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 INFO: main: starting shard=10 2024-03-04T17:45:17.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.868+0000 7fe639769980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:17.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.872+0000 7fe639769980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:17.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.872+0000 7fe639769980 20 INFO: main: done 2024-03-04T17:45:17.876 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.872+0000 7fe4edfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:17.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.876+0000 7fe639769980 20 remove_watcher() i=0 2024-03-04T17:45:17.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.876+0000 7fe639769980 2 removed watcher, disabling cache 2024-03-04T17:45:17.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.880+0000 7fe639769980 20 remove_watcher() i=1 2024-03-04T17:45:17.885 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.884+0000 7fe639769980 20 remove_watcher() i=2 2024-03-04T17:45:17.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.884+0000 7fe639769980 20 remove_watcher() i=3 2024-03-04T17:45:17.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.888+0000 7fe639769980 20 remove_watcher() i=4 2024-03-04T17:45:17.893 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.888+0000 7fe639769980 20 remove_watcher() i=5 2024-03-04T17:45:17.896 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.892+0000 7fe639769980 20 remove_watcher() i=6 2024-03-04T17:45:17.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:17.896+0000 7fe639769980 20 remove_watcher() i=7 2024-03-04T17:45:17.919 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:17.919 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:17.958 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:17.958 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:18.127 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:18.127 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-03-04T17:45:18.164 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:18.164 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:18.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.204+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 realm 2024-03-04T17:45:18.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:18.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:18.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:18.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.208+0000 7f71ecacd980 20 searching for the correct realm 2024-03-04T17:45:18.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.216+0000 7f71e9de4640 0 -- 172.21.15.47:0/2309480485 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f71e0104040 msgr2=0x7f71e00f6410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:18.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.224+0000 7f71ea5e5640 0 -- 172.21.15.47:0/2309480485 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f71e401f040 msgr2=0x7f71e4058a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:18.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:18.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:18.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:18.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:18.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:18.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:18.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 zone default found 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 4 Realm: () 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 zonegroup default 2024-03-04T17:45:18.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 started sync module instance, tier type = 2024-03-04T17:45:18.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.232+0000 7f71ecacd980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:18.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.256+0000 7f71ecacd980 20 add_watcher() i=0 2024-03-04T17:45:18.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.256+0000 7f71ecacd980 20 add_watcher() i=1 2024-03-04T17:45:18.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.256+0000 7f71ecacd980 20 add_watcher() i=2 2024-03-04T17:45:18.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.256+0000 7f71ecacd980 20 add_watcher() i=3 2024-03-04T17:45:18.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.256+0000 7f71ecacd980 20 add_watcher() i=4 2024-03-04T17:45:18.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.256+0000 7f71ecacd980 20 add_watcher() i=5 2024-03-04T17:45:18.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.256+0000 7f71ecacd980 20 add_watcher() i=6 2024-03-04T17:45:18.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.256+0000 7f71ecacd980 20 add_watcher() i=7 2024-03-04T17:45:18.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.256+0000 7f71ecacd980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:18.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.260+0000 7f70a0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:18.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.260+0000 7f71ecacd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:18.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.260+0000 7f71ecacd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:18.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.260+0000 7f71ecacd980 5 note: GC not initialized 2024-03-04T17:45:18.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.260+0000 7f701b7fe640 20 reqs_thread_entry: start 2024-03-04T17:45:18.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.260+0000 7f71ecacd980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:18.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.260+0000 7f71ecacd980 20 Filter name: none 2024-03-04T17:45:18.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.260+0000 7f70197fa640 20 reqs_thread_entry: start 2024-03-04T17:45:18.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.260+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:18.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:18.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:18.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:18.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 INFO: main: starting shard=0 2024-03-04T17:45:18.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:18.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 INFO: main: starting shard=1 2024-03-04T17:45:18.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:18.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T17:45:18.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 INFO: main: starting shard=2 2024-03-04T17:45:18.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.264+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:18.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: starting shard=3 2024-03-04T17:45:18.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:18.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: starting shard=4 2024-03-04T17:45:18.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:18.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: starting shard=5 2024-03-04T17:45:18.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:18.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: starting shard=6 2024-03-04T17:45:18.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:18.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 INFO: main: starting shard=7 2024-03-04T17:45:18.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.268+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:18.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 INFO: main: starting shard=8 2024-03-04T17:45:18.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:18.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 INFO: main: starting shard=9 2024-03-04T17:45:18.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:18.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 INFO: main: starting shard=10 2024-03-04T17:45:18.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:18.277 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.277 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.272+0000 7f71ecacd980 20 INFO: main: done 2024-03-04T17:45:18.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.276+0000 7f70a0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:18.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.280+0000 7f71ecacd980 20 remove_watcher() i=0 2024-03-04T17:45:18.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.280+0000 7f71ecacd980 2 removed watcher, disabling cache 2024-03-04T17:45:18.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.280+0000 7f71ecacd980 20 remove_watcher() i=1 2024-03-04T17:45:18.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.284+0000 7f71ecacd980 20 remove_watcher() i=2 2024-03-04T17:45:18.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.284+0000 7f71ecacd980 20 remove_watcher() i=3 2024-03-04T17:45:18.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.288+0000 7f71ecacd980 20 remove_watcher() i=4 2024-03-04T17:45:18.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.288+0000 7f71ecacd980 20 remove_watcher() i=5 2024-03-04T17:45:18.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.292+0000 7f71ecacd980 20 remove_watcher() i=6 2024-03-04T17:45:18.299 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.296+0000 7f71ecacd980 20 remove_watcher() i=7 2024-03-04T17:45:18.316 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:18.316 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:18.341 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:18.342 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:18.459 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:18.459 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-03-04T17:45:18.496 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:18.496 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:18.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.536+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 20 realm 2024-03-04T17:45:18.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:18.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:18.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.540+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.544+0000 7f6e02dcb980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:18.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.544+0000 7f6e02dcb980 20 searching for the correct realm 2024-03-04T17:45:18.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.548+0000 7f6e000e2640 0 -- 172.21.15.47:0/3271849935 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f6df40e5060 msgr2=0x7f6df411f130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:18.559 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.556+0000 7f6e010e4640 0 -- 172.21.15.47:0/3271849935 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x562ec570f600 msgr2=0x562ec5711aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:18.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:18.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:18.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:18.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:18.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:18.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:18.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:18.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:18.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 zone default found 2024-03-04T17:45:18.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 4 Realm: () 2024-03-04T17:45:18.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:18.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:18.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:18.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 zonegroup default 2024-03-04T17:45:18.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.564+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.568+0000 7f6e02dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.568+0000 7f6e02dcb980 20 started sync module instance, tier type = 2024-03-04T17:45:18.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.568+0000 7f6e02dcb980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:18.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.588+0000 7f6e02dcb980 20 add_watcher() i=0 2024-03-04T17:45:18.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.588+0000 7f6e02dcb980 20 add_watcher() i=1 2024-03-04T17:45:18.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.588+0000 7f6e02dcb980 20 add_watcher() i=2 2024-03-04T17:45:18.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.588+0000 7f6e02dcb980 20 add_watcher() i=3 2024-03-04T17:45:18.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.588+0000 7f6e02dcb980 20 add_watcher() i=4 2024-03-04T17:45:18.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.588+0000 7f6e02dcb980 20 add_watcher() i=5 2024-03-04T17:45:18.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.588+0000 7f6e02dcb980 20 add_watcher() i=6 2024-03-04T17:45:18.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.592+0000 7f6e02dcb980 20 add_watcher() i=7 2024-03-04T17:45:18.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.592+0000 7f6e02dcb980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:18.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.592+0000 7f6cb6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:18.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:18.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:18.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 5 note: GC not initialized 2024-03-04T17:45:18.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6cb5fe3640 20 reqs_thread_entry: start 2024-03-04T17:45:18.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:18.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 Filter name: none 2024-03-04T17:45:18.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6c277fe640 20 reqs_thread_entry: start 2024-03-04T17:45:18.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:18.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:18.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:18.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:18.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 INFO: main: starting shard=0 2024-03-04T17:45:18.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.596+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:18.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: starting shard=1 2024-03-04T17:45:18.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:18.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: starting shard=2 2024-03-04T17:45:18.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:18.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: starting shard=3 2024-03-04T17:45:18.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:18.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: starting shard=4 2024-03-04T17:45:18.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:18.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 INFO: main: starting shard=5 2024-03-04T17:45:18.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.600+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:18.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 INFO: main: starting shard=6 2024-03-04T17:45:18.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:18.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 INFO: main: starting shard=7 2024-03-04T17:45:18.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:18.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 INFO: main: starting shard=8 2024-03-04T17:45:18.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:18.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 INFO: main: starting shard=9 2024-03-04T17:45:18.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.604+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:18.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.608+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:18.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.608+0000 7f6e02dcb980 20 INFO: main: starting shard=10 2024-03-04T17:45:18.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.608+0000 7f6e02dcb980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:18.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.608+0000 7f6e02dcb980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T17:45:18.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.608+0000 7f6e02dcb980 20 INFO: main: done 2024-03-04T17:45:18.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.608+0000 7f6cb6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:18.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.616+0000 7f6e02dcb980 20 remove_watcher() i=0 2024-03-04T17:45:18.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.616+0000 7f6e02dcb980 2 removed watcher, disabling cache 2024-03-04T17:45:18.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.616+0000 7f6e02dcb980 20 remove_watcher() i=1 2024-03-04T17:45:18.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.620+0000 7f6e02dcb980 20 remove_watcher() i=2 2024-03-04T17:45:18.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.620+0000 7f6e02dcb980 20 remove_watcher() i=3 2024-03-04T17:45:18.627 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.624+0000 7f6e02dcb980 20 remove_watcher() i=4 2024-03-04T17:45:18.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.628+0000 7f6e02dcb980 20 remove_watcher() i=5 2024-03-04T17:45:18.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.628+0000 7f6e02dcb980 20 remove_watcher() i=6 2024-03-04T17:45:18.635 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:18.632+0000 7f6e02dcb980 20 remove_watcher() i=7 2024-03-04T17:45:18.653 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:18.653 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:18.676 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-03-04T17:45:19.101 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-03-04T17:45:19.516 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: bucket check unlinked finds unlistable entries 2024-03-04T17:45:19.517 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:19.517 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-03-04T17:45:19.551 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:19.551 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:19.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.588+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 20 realm 2024-03-04T17:45:19.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:19.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:19.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.592+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.596+0000 7f85fad02980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:19.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.596+0000 7f85fad02980 20 searching for the correct realm 2024-03-04T17:45:19.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.600+0000 7f85f901b640 0 -- 172.21.15.47:0/896617581 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f85e0053fd0 msgr2=0x7f85e0082240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:19.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.608+0000 7f85f881a640 0 -- 172.21.15.47:0/896617581 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f85f0035040 msgr2=0x7f85f0046950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:19.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:19.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:19.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:19.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:19.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:19.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:19.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:19.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:19.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 zone default found 2024-03-04T17:45:19.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 4 Realm: () 2024-03-04T17:45:19.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:19.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:19.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:19.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 zonegroup default 2024-03-04T17:45:19.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.616+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.620+0000 7f85fad02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.620+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.620+0000 7f85fad02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.620+0000 7f85fad02980 20 started sync module instance, tier type = 2024-03-04T17:45:19.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.620+0000 7f85fad02980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:19.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.640+0000 7f85fad02980 20 add_watcher() i=0 2024-03-04T17:45:19.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.640+0000 7f85fad02980 20 add_watcher() i=1 2024-03-04T17:45:19.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.640+0000 7f85fad02980 20 add_watcher() i=2 2024-03-04T17:45:19.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.640+0000 7f85fad02980 20 add_watcher() i=3 2024-03-04T17:45:19.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.640+0000 7f85fad02980 20 add_watcher() i=4 2024-03-04T17:45:19.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.640+0000 7f85fad02980 20 add_watcher() i=5 2024-03-04T17:45:19.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.640+0000 7f85fad02980 20 add_watcher() i=6 2024-03-04T17:45:19.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.644+0000 7f85fad02980 20 add_watcher() i=7 2024-03-04T17:45:19.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.644+0000 7f85fad02980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:19.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f84aefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:19.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f85fad02980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:19.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f85fad02980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:19.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f85fad02980 5 note: GC not initialized 2024-03-04T17:45:19.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f84adfe3640 20 reqs_thread_entry: start 2024-03-04T17:45:19.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f85fad02980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:19.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f85fad02980 20 Filter name: none 2024-03-04T17:45:19.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f841f7fe640 20 reqs_thread_entry: start 2024-03-04T17:45:19.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f85fad02980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:19.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.648+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:19.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:19.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:19.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 1 NOTICE: finished shard 0 (0 entries found) 2024-03-04T17:45:19.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:19.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:19.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 1 NOTICE: finished shard 2 (0 entries found) 2024-03-04T17:45:19.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.652+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:19.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.656+0000 7f85fad02980 1 NOTICE: finished shard 3 (0 entries found) 2024-03-04T17:45:19.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.656+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:19.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.656+0000 7f85fad02980 1 NOTICE: finished shard 4 (0 entries found) 2024-03-04T17:45:19.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.656+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:19.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.656+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:19.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.656+0000 7f85fad02980 1 NOTICE: finished shard 6 (0 entries found) 2024-03-04T17:45:19.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.656+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:19.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.660+0000 7f85fad02980 1 NOTICE: finished shard 7 (0 entries found) 2024-03-04T17:45:19.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.660+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:19.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.660+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:19.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.660+0000 7f85fad02980 1 NOTICE: finished shard 9 (0 entries found) 2024-03-04T17:45:19.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.660+0000 7f85fad02980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:19.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.660+0000 7f85fad02980 1 NOTICE: finished shard 1 (1 entries found) 2024-03-04T17:45:19.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.660+0000 7f85fad02980 1 NOTICE: finished shard 5 (1 entries found) 2024-03-04T17:45:19.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.660+0000 7f85fad02980 1 NOTICE: finished shard 8 (1 entries found) 2024-03-04T17:45:19.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.660+0000 7f85fad02980 1 NOTICE: finished shard 10 (1 entries found) 2024-03-04T17:45:19.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.664+0000 7f85fad02980 1 NOTICE: finished all shards (4 entries found) 2024-03-04T17:45:19.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.664+0000 7f84aefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:19.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.668+0000 7f85fad02980 20 remove_watcher() i=0 2024-03-04T17:45:19.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.668+0000 7f85fad02980 2 removed watcher, disabling cache 2024-03-04T17:45:19.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.672+0000 7f85fad02980 20 remove_watcher() i=1 2024-03-04T17:45:19.676 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.672+0000 7f85fad02980 20 remove_watcher() i=2 2024-03-04T17:45:19.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.676+0000 7f85fad02980 20 remove_watcher() i=3 2024-03-04T17:45:19.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.676+0000 7f85fad02980 20 remove_watcher() i=4 2024-03-04T17:45:19.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.680+0000 7f85fad02980 20 remove_watcher() i=5 2024-03-04T17:45:19.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.684+0000 7f85fad02980 20 remove_watcher() i=6 2024-03-04T17:45:19.689 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:19.684+0000 7f85fad02980 20 remove_watcher() i=7 2024-03-04T17:45:19.704 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: unlinked entries are not listable 2024-03-04T17:45:19.705 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:19.707 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.707 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:19.882 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: GET returns 404 for unlinked entry keys that have no other versions 2024-03-04T17:45:19.882 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:19.887 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.887 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:19.985 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.985 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.075 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: bucket check unlinked fixes unlistable entries 2024-03-04T17:45:20.075 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:20.075 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-03-04T17:45:20.116 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:20.116 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:20.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.156+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 realm 2024-03-04T17:45:20.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:20.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:20.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:20.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.160+0000 7f4165653980 20 searching for the correct realm 2024-03-04T17:45:20.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.168+0000 7f416316b640 0 -- 172.21.15.47:0/3054856203 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f4154085050 msgr2=0x7f4154054400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:20.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.172+0000 7f416316b640 0 -- 172.21.15.47:0/3054856203 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f415c0f3c00 msgr2=0x7f415c0f8a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:20.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:20.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:20.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:20.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:20.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:20.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:20.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:20.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.180+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:20.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 zone default found 2024-03-04T17:45:20.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 4 Realm: () 2024-03-04T17:45:20.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:20.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:20.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:20.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 zonegroup default 2024-03-04T17:45:20.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 started sync module instance, tier type = 2024-03-04T17:45:20.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.184+0000 7f4165653980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:20.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.204+0000 7f4165653980 20 add_watcher() i=0 2024-03-04T17:45:20.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.204+0000 7f4165653980 20 add_watcher() i=1 2024-03-04T17:45:20.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.204+0000 7f4165653980 20 add_watcher() i=2 2024-03-04T17:45:20.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.204+0000 7f4165653980 20 add_watcher() i=3 2024-03-04T17:45:20.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.204+0000 7f4165653980 20 add_watcher() i=4 2024-03-04T17:45:20.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.204+0000 7f4165653980 20 add_watcher() i=5 2024-03-04T17:45:20.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.204+0000 7f4165653980 20 add_watcher() i=6 2024-03-04T17:45:20.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.208+0000 7f4165653980 20 add_watcher() i=7 2024-03-04T17:45:20.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.208+0000 7f4165653980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:20.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.208+0000 7f4019fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:20.214 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:20.214 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:20.214 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 5 note: GC not initialized 2024-03-04T17:45:20.214 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4018fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:20.214 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:20.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 Filter name: none 2024-03-04T17:45:20.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f3f927fc640 20 reqs_thread_entry: start 2024-03-04T17:45:20.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:20.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:20.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:20.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:20.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:20.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:20.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:20.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:20.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:20.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:20.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:20.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:20.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 1 NOTICE: finished shard 0 (0 entries removed) 2024-03-04T17:45:20.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.212+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:20.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.216+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:20.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.216+0000 7f4165653980 1 NOTICE: finished shard 2 (0 entries removed) 2024-03-04T17:45:20.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.216+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:20.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.216+0000 7f4165653980 1 NOTICE: finished shard 3 (0 entries removed) 2024-03-04T17:45:20.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.216+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:20.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.216+0000 7f4165653980 1 NOTICE: finished shard 4 (0 entries removed) 2024-03-04T17:45:20.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.216+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:20.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.216+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:20.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.220+0000 7f4165653980 1 NOTICE: finished shard 6 (0 entries removed) 2024-03-04T17:45:20.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.220+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:20.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.220+0000 7f4165653980 1 NOTICE: finished shard 7 (0 entries removed) 2024-03-04T17:45:20.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.220+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:20.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.220+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:20.225 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.220+0000 7f4165653980 1 NOTICE: finished shard 9 (0 entries removed) 2024-03-04T17:45:20.225 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.220+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:20.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.224+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:20.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.224+0000 7f4165653980 20 get_obj_state: octx=0x558e555aa950 obj=check-bucket:e state=0x558e558569f8 s->prefetch_data=0 2024-03-04T17:45:20.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.224+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:20.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.224+0000 7f4165653980 20 get_obj_state: octx=0x558e555bbbf0 obj=check-bucket:d state=0x558e55857f28 s->prefetch_data=0 2024-03-04T17:45:20.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.224+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:20.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.224+0000 7f4165653980 20 get_obj_state: octx=0x558e55859740 obj=check-bucket:c state=0x558e55853928 s->prefetch_data=0 2024-03-04T17:45:20.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.224+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:20.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 get_obj_state: octx=0x558e5585ba10 obj=check-bucket:f state=0x558e55854488 s->prefetch_data=0 2024-03-04T17:45:20.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 get_obj_state: setting s->obj_tag to shsoe2onwaqj9p5tfvomxt5e3hvb1f21 2024-03-04T17:45:20.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 get_obj_state_impl: setting s->olh_tag to d6v98kcos57zggydewkk6dspwwdbobnd 2024-03-04T17:45:20.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=d6v98kcos57zggydewkk6dspwwdbobnd 2024-03-04T17:45:20.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 get_obj_state: setting s->obj_tag to ixyr3rcvls8k0m4ea753mvuv3boh21fy 2024-03-04T17:45:20.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 get_obj_state_impl: setting s->olh_tag to 2iygikqtxg0fyltrw72jf6cnysli3jzg 2024-03-04T17:45:20.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=2iygikqtxg0fyltrw72jf6cnysli3jzg 2024-03-04T17:45:20.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 get_obj_state: setting s->obj_tag to ncllb1gi35iifd5o18jy11ozauzav507 2024-03-04T17:45:20.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 get_obj_state_impl: setting s->olh_tag to s9y11ad4sqn1tx7eoth4dsodgkgdhkzs 2024-03-04T17:45:20.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=s9y11ad4sqn1tx7eoth4dsodgkgdhkzs 2024-03-04T17:45:20.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 get_obj_state: setting s->obj_tag to uyusjm5m59l157trxanu1fx3epq3xifl 2024-03-04T17:45:20.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 get_obj_state_impl: setting s->olh_tag to 0dkspqu8nupgi5lt8vul2a0pzuyi90ev 2024-03-04T17:45:20.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=0dkspqu8nupgi5lt8vul2a0pzuyi90ev 2024-03-04T17:45:20.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:20.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.228+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:20.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:20.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:20.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:20.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-03-04T17:45:20.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 olh_log_entry: epoch=2 op=3 key=e[CRC2RBxcrgPzAItBeEixXQ5uliv2WI6] 2024-03-04T17:45:20.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 get_obj_state: octx=0x558e555aa950 obj=check-bucket:_:CRC2RBxcrgPzAItBeEixXQ5uliv2WI6_e state=0x558e558fcd48 s->prefetch_data=0 2024-03-04T17:45:20.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:20.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 10 manifest: total_size = 22 2024-03-04T17:45:20.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.8771418471770812671 2024-03-04T17:45:20.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 get_obj_state: octx=0x558e555aa950 obj=check-bucket:_:CRC2RBxcrgPzAItBeEixXQ5uliv2WI6_e state=0x558e558fcd48 s->prefetch_data=0 2024-03-04T17:45:20.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 prepare_atomic_modification: state is not atomic. state=0x558e558fcd48 2024-03-04T17:45:20.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.232+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 olh_log_entry: epoch=3 op=3 key=d[kjslwyCP24UcncbFVKuYd-yQwIAYmxj] 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 get_obj_state: octx=0x558e555bbbf0 obj=check-bucket:_:kjslwyCP24UcncbFVKuYd-yQwIAYmxj_d state=0x558e55901858 s->prefetch_data=0 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 10 manifest: total_size = 12 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.12755135558076720222 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 get_obj_state: octx=0x558e555bbbf0 obj=check-bucket:_:kjslwyCP24UcncbFVKuYd-yQwIAYmxj_d state=0x558e55901858 s->prefetch_data=0 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 prepare_atomic_modification: state is not atomic. state=0x558e55901858 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 olh_log_entry: epoch=3 op=3 key=c[dNu3Lz8imn0D-DtZ-8tILnoEYRedy6-] 2024-03-04T17:45:20.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 get_obj_state: octx=0x558e55859740 obj=check-bucket:_:dNu3Lz8imn0D-DtZ-8tILnoEYRedy6-_c state=0x558e5590db38 s->prefetch_data=0 2024-03-04T17:45:20.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 10 manifest: total_size = 22 2024-03-04T17:45:20.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.13619098041641388012 2024-03-04T17:45:20.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 get_obj_state: octx=0x558e55859740 obj=check-bucket:_:dNu3Lz8imn0D-DtZ-8tILnoEYRedy6-_c state=0x558e5590db38 s->prefetch_data=0 2024-03-04T17:45:20.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 prepare_atomic_modification: state is not atomic. state=0x558e5590db38 2024-03-04T17:45:20.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:20.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.236+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:20.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.240+0000 7f4165653980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-03-04T17:45:20.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.240+0000 7f4165653980 20 olh_log_entry: epoch=2 op=3 key=f[NoinvPLoDtOggmOMxatC7.UKP7.yg-S] 2024-03-04T17:45:20.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.240+0000 7f4165653980 20 get_obj_state: octx=0x558e5585ba10 obj=check-bucket:_:NoinvPLoDtOggmOMxatC7.UKP7.yg-S_f state=0x558e55913f88 s->prefetch_data=0 2024-03-04T17:45:20.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.240+0000 7f4165653980 10 manifest: total_size = 9 2024-03-04T17:45:20.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.240+0000 7f4165653980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.15323787524170484708 2024-03-04T17:45:20.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.240+0000 7f4165653980 20 get_obj_state: octx=0x558e5585ba10 obj=check-bucket:_:NoinvPLoDtOggmOMxatC7.UKP7.yg-S_f state=0x558e55913f88 s->prefetch_data=0 2024-03-04T17:45:20.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.240+0000 7f4165653980 20 prepare_atomic_modification: state is not atomic. state=0x558e55913f88 2024-03-04T17:45:20.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.240+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:20.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.244+0000 7f4046ffd640 20 handle_completion(): completion ok for obj=e[CRC2RBxcrgPzAItBeEixXQ5uliv2WI6] 2024-03-04T17:45:20.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.248+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:20.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.248+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:20.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.252+0000 7f4165653980 20 get_obj_state: octx=0x558e555aa950 obj=check-bucket:e state=0x558e558569f8 s->prefetch_data=0 2024-03-04T17:45:20.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.252+0000 7f4165653980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=d6v98kcos57zggydewkk6dspwwdbobnd 2024-03-04T17:45:20.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.252+0000 7f40467fc640 20 handle_completion(): completion ok for obj=c[dNu3Lz8imn0D-DtZ-8tILnoEYRedy6-] 2024-03-04T17:45:20.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.252+0000 7f4046ffd640 20 handle_completion(): completion ok for obj=d[kjslwyCP24UcncbFVKuYd-yQwIAYmxj] 2024-03-04T17:45:20.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.252+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:20.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.252+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:20.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.256+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:20.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.256+0000 7f4165653980 1 NOTICE: finished shard 5 (1 entries removed) 2024-03-04T17:45:20.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.256+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:20.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.256+0000 7f4165653980 1 NOTICE: finished shard 8 (1 entries removed) 2024-03-04T17:45:20.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.256+0000 7f4046ffd640 20 handle_completion(): completion ok for obj=f[NoinvPLoDtOggmOMxatC7.UKP7.yg-S] 2024-03-04T17:45:20.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.260+0000 7f4165653980 20 get_obj_state: octx=0x558e5585ba10 obj=check-bucket:f state=0x558e55854488 s->prefetch_data=0 2024-03-04T17:45:20.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.260+0000 7f4165653980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=0dkspqu8nupgi5lt8vul2a0pzuyi90ev 2024-03-04T17:45:20.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.264+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:20.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.264+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:20.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.264+0000 7f4165653980 1 NOTICE: finished shard 1 (1 entries removed) 2024-03-04T17:45:20.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.268+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:20.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.272+0000 7f4165653980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:20.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.272+0000 7f4165653980 1 NOTICE: finished shard 10 (1 entries removed) 2024-03-04T17:45:20.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.272+0000 7f4165653980 1 NOTICE: finished all shards (4 entries removed) 2024-03-04T17:45:20.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.272+0000 7f4019fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:20.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.276+0000 7f4165653980 20 remove_watcher() i=0 2024-03-04T17:45:20.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.276+0000 7f4165653980 2 removed watcher, disabling cache 2024-03-04T17:45:20.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.280+0000 7f4165653980 20 remove_watcher() i=1 2024-03-04T17:45:20.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.280+0000 7f4165653980 20 remove_watcher() i=2 2024-03-04T17:45:20.287 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.284+0000 7f4165653980 20 remove_watcher() i=3 2024-03-04T17:45:20.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.284+0000 7f4165653980 20 remove_watcher() i=4 2024-03-04T17:45:20.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.288+0000 7f4165653980 20 remove_watcher() i=5 2024-03-04T17:45:20.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.292+0000 7f4165653980 20 remove_watcher() i=6 2024-03-04T17:45:20.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.292+0000 7f4165653980 20 remove_watcher() i=7 2024-03-04T17:45:20.313 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.314 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.334 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.334 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.358 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.358 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.386 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.386 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.428 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: bucket check unlinked does not affect normal entries 2024-03-04T17:45:20.428 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:20.430 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.430 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.561 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.561 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.619 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.619 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.667 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.667 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.731 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.731 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:20.793 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: bucket check unlinked does not find new unlistable entries after fix 2024-03-04T17:45:20.793 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:20.793 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-03-04T17:45:20.832 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:20.833 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:20.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.872+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.876+0000 7f2809c18980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.876+0000 7f2809c18980 20 realm 2024-03-04T17:45:20.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.876+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.876+0000 7f2809c18980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.876+0000 7f2809c18980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:20.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.876+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.876+0000 7f2809c18980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.876+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.880+0000 7f2809c18980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:20.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.880+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.880+0000 7f2809c18980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:20.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.880+0000 7f2809c18980 20 searching for the correct realm 2024-03-04T17:45:20.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:20.884+0000 7f2806f2f640 0 -- 172.21.15.47:0/99610072 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f2800009250 msgr2=0x7f28000b8040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:21.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.096+0000 7f2807f31640 0 -- 172.21.15.47:0/99610072 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x5586ec33c7f0 msgr2=0x5586ec33ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:21.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:21.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:21.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:21.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:21.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:21.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:21.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:21.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:21.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 zone default found 2024-03-04T17:45:21.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 4 Realm: () 2024-03-04T17:45:21.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:21.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:21.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:21.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 zonegroup default 2024-03-04T17:45:21.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.112+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.116+0000 7f2809c18980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.116+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.116+0000 7f2809c18980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.116+0000 7f2809c18980 20 started sync module instance, tier type = 2024-03-04T17:45:21.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.116+0000 7f2809c18980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:21.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.144+0000 7f2809c18980 20 add_watcher() i=0 2024-03-04T17:45:21.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.144+0000 7f2809c18980 20 add_watcher() i=1 2024-03-04T17:45:21.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.144+0000 7f2809c18980 20 add_watcher() i=2 2024-03-04T17:45:21.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.144+0000 7f2809c18980 20 add_watcher() i=3 2024-03-04T17:45:21.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.144+0000 7f2809c18980 20 add_watcher() i=4 2024-03-04T17:45:21.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.144+0000 7f2809c18980 20 add_watcher() i=5 2024-03-04T17:45:21.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.144+0000 7f2809c18980 20 add_watcher() i=6 2024-03-04T17:45:21.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.144+0000 7f2809c18980 20 add_watcher() i=7 2024-03-04T17:45:21.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.144+0000 7f2809c18980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:21.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.148+0000 7f26bdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:21.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:21.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 5 note: GC not initialized 2024-03-04T17:45:21.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f26bcfe1640 20 reqs_thread_entry: start 2024-03-04T17:45:21.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:21.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 20 Filter name: none 2024-03-04T17:45:21.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f26367fc640 20 reqs_thread_entry: start 2024-03-04T17:45:21.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:21.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:21.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:21.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:21.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:21.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.152+0000 7f2809c18980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:21.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:21.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:21.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:21.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:21.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:21.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:21.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 1 NOTICE: finished shard 0 (0 entries found) 2024-03-04T17:45:21.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:21.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 1 NOTICE: finished shard 1 (0 entries found) 2024-03-04T17:45:21.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:21.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 1 NOTICE: finished shard 2 (0 entries found) 2024-03-04T17:45:21.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:21.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 1 NOTICE: finished shard 3 (0 entries found) 2024-03-04T17:45:21.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.156+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:21.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.160+0000 7f2809c18980 1 NOTICE: finished shard 4 (0 entries found) 2024-03-04T17:45:21.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.160+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:21.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.160+0000 7f2809c18980 1 NOTICE: finished shard 5 (0 entries found) 2024-03-04T17:45:21.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.160+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:21.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.160+0000 7f2809c18980 1 NOTICE: finished shard 6 (0 entries found) 2024-03-04T17:45:21.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.160+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:21.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.160+0000 7f2809c18980 1 NOTICE: finished shard 7 (0 entries found) 2024-03-04T17:45:21.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.160+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:21.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.164+0000 7f2809c18980 1 NOTICE: finished shard 8 (0 entries found) 2024-03-04T17:45:21.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.164+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:21.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.164+0000 7f2809c18980 1 NOTICE: finished shard 9 (0 entries found) 2024-03-04T17:45:21.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.164+0000 7f2809c18980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:21.168 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.164+0000 7f2809c18980 1 NOTICE: finished shard 10 (0 entries found) 2024-03-04T17:45:21.168 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.164+0000 7f2809c18980 1 NOTICE: finished all shards (0 entries found) 2024-03-04T17:45:21.170 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.168+0000 7f26bdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.172+0000 7f2809c18980 20 remove_watcher() i=0 2024-03-04T17:45:21.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.172+0000 7f2809c18980 2 removed watcher, disabling cache 2024-03-04T17:45:21.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.172+0000 7f2809c18980 20 remove_watcher() i=1 2024-03-04T17:45:21.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.176+0000 7f2809c18980 20 remove_watcher() i=2 2024-03-04T17:45:21.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.180+0000 7f2809c18980 20 remove_watcher() i=3 2024-03-04T17:45:21.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.180+0000 7f2809c18980 20 remove_watcher() i=4 2024-03-04T17:45:21.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.184+0000 7f2809c18980 20 remove_watcher() i=5 2024-03-04T17:45:21.189 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.188+0000 7f2809c18980 20 remove_watcher() i=6 2024-03-04T17:45:21.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.188+0000 7f2809c18980 20 remove_watcher() i=7 2024-03-04T17:45:21.212 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:21.213 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:21.485 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:21.485 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:21.613 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-03-04T17:45:21.650 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:21.650 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:21.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.692+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 realm 2024-03-04T17:45:21.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:21.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:21.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:21.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.696+0000 7f2f65c0d980 20 searching for the correct realm 2024-03-04T17:45:21.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.704+0000 7f2f62f24640 0 -- 172.21.15.47:0/1529594245 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f2f580c5740 msgr2=0x7f2f58117030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:21.921 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.916+0000 7f2f63f26640 0 -- 172.21.15.47:0/1529594245 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x5623f7556770 msgr2=0x5623f7558c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:21.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.928+0000 7f2f65c0d980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:21.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.928+0000 7f2f65c0d980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:21.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.928+0000 7f2f65c0d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:21.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.928+0000 7f2f65c0d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:21.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.928+0000 7f2f65c0d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:21.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.928+0000 7f2f65c0d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:21.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.928+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:21.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:21.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 zone default found 2024-03-04T17:45:21.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 4 Realm: () 2024-03-04T17:45:21.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:21.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:21.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:21.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 zonegroup default 2024-03-04T17:45:21.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.932+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.936+0000 7f2f65c0d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.936+0000 7f2f65c0d980 20 started sync module instance, tier type = 2024-03-04T17:45:21.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.936+0000 7f2f65c0d980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:21.964 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.960+0000 7f2f65c0d980 20 add_watcher() i=0 2024-03-04T17:45:21.964 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.960+0000 7f2f65c0d980 20 add_watcher() i=1 2024-03-04T17:45:21.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.960+0000 7f2f65c0d980 20 add_watcher() i=2 2024-03-04T17:45:21.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.960+0000 7f2f65c0d980 20 add_watcher() i=3 2024-03-04T17:45:21.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.960+0000 7f2f65c0d980 20 add_watcher() i=4 2024-03-04T17:45:21.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.960+0000 7f2f65c0d980 20 add_watcher() i=5 2024-03-04T17:45:21.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.960+0000 7f2f65c0d980 20 add_watcher() i=6 2024-03-04T17:45:21.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.964+0000 7f2f65c0d980 20 add_watcher() i=7 2024-03-04T17:45:21.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.964+0000 7f2f65c0d980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:21.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2e19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:21.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:21.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 5 note: GC not initialized 2024-03-04T17:45:21.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2e18fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:21.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:21.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 Filter name: none 2024-03-04T17:45:21.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2d927fc640 20 reqs_thread_entry: start 2024-03-04T17:45:21.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:21.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:21.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:21.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:21.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 INFO: main: starting shard=0 2024-03-04T17:45:21.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.968+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:21.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 INFO: main: starting shard=1 2024-03-04T17:45:21.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:21.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 INFO: main: starting shard=2 2024-03-04T17:45:21.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:21.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 INFO: main: starting shard=3 2024-03-04T17:45:21.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:21.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 INFO: main: starting shard=4 2024-03-04T17:45:21.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.972+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:21.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: starting shard=5 2024-03-04T17:45:21.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:21.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: starting shard=6 2024-03-04T17:45:21.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:21.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: starting shard=7 2024-03-04T17:45:21.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:21.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: starting shard=8 2024-03-04T17:45:21.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:21.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-03-04T17:45:21.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 INFO: main: starting shard=9 2024-03-04T17:45:21.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.976+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:21.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.980+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.980+0000 7f2f65c0d980 20 INFO: main: starting shard=10 2024-03-04T17:45:21.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.980+0000 7f2f65c0d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:21.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.980+0000 7f2f65c0d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:21.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.980+0000 7f2f65c0d980 20 INFO: main: done 2024-03-04T17:45:21.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.980+0000 7f2e19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.988+0000 7f2f65c0d980 20 remove_watcher() i=0 2024-03-04T17:45:21.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.988+0000 7f2f65c0d980 2 removed watcher, disabling cache 2024-03-04T17:45:21.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.988+0000 7f2f65c0d980 20 remove_watcher() i=1 2024-03-04T17:45:21.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.992+0000 7f2f65c0d980 20 remove_watcher() i=2 2024-03-04T17:45:21.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.992+0000 7f2f65c0d980 20 remove_watcher() i=3 2024-03-04T17:45:21.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:21.996+0000 7f2f65c0d980 20 remove_watcher() i=4 2024-03-04T17:45:22.002 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.000+0000 7f2f65c0d980 20 remove_watcher() i=5 2024-03-04T17:45:22.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.000+0000 7f2f65c0d980 20 remove_watcher() i=6 2024-03-04T17:45:22.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.004+0000 7f2f65c0d980 20 remove_watcher() i=7 2024-03-04T17:45:22.027 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-03-04T17:45:22.061 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:22.061 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:22.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.100+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 realm 2024-03-04T17:45:22.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:22.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:22.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.104+0000 7f5c61b0e980 20 searching for the correct realm 2024-03-04T17:45:22.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.112+0000 7f5c5ee25640 0 -- 172.21.15.47:0/2115108828 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f5c5400a9c0 msgr2=0x7f5c540798f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:22.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.324+0000 7f5c5fe27640 0 -- 172.21.15.47:0/2115108828 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55927d92cad0 msgr2=0x55927d92eee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:22.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.336+0000 7f5c61b0e980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:22.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.336+0000 7f5c61b0e980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:22.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.336+0000 7f5c61b0e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:22.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.336+0000 7f5c61b0e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:22.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.336+0000 7f5c61b0e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:22.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.336+0000 7f5c61b0e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:22.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.336+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:22.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 zone default found 2024-03-04T17:45:22.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 4 Realm: () 2024-03-04T17:45:22.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:22.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:22.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:22.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 zonegroup default 2024-03-04T17:45:22.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.340+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.344+0000 7f5c61b0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.344+0000 7f5c61b0e980 20 started sync module instance, tier type = 2024-03-04T17:45:22.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.344+0000 7f5c61b0e980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:22.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.372+0000 7f5c61b0e980 20 add_watcher() i=0 2024-03-04T17:45:22.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.372+0000 7f5c61b0e980 20 add_watcher() i=1 2024-03-04T17:45:22.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.372+0000 7f5c61b0e980 20 add_watcher() i=2 2024-03-04T17:45:22.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.372+0000 7f5c61b0e980 20 add_watcher() i=3 2024-03-04T17:45:22.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.372+0000 7f5c61b0e980 20 add_watcher() i=4 2024-03-04T17:45:22.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.372+0000 7f5c61b0e980 20 add_watcher() i=5 2024-03-04T17:45:22.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.372+0000 7f5c61b0e980 20 add_watcher() i=6 2024-03-04T17:45:22.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.376+0000 7f5c61b0e980 20 add_watcher() i=7 2024-03-04T17:45:22.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.376+0000 7f5c61b0e980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:22.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.380+0000 7f5b15fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5c61b0e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:22.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5c61b0e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:22.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5c61b0e980 5 note: GC not initialized 2024-03-04T17:45:22.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5b14fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:22.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5c61b0e980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:22.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5c61b0e980 20 Filter name: none 2024-03-04T17:45:22.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5a8e7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:22.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5c61b0e980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:22.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.384+0000 7f5c61b0e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:22.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:22.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:22.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 INFO: main: starting shard=0 2024-03-04T17:45:22.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:22.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 INFO: main: starting shard=1 2024-03-04T17:45:22.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:22.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 INFO: main: starting shard=2 2024-03-04T17:45:22.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:22.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 INFO: main: starting shard=3 2024-03-04T17:45:22.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.388+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:22.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.392+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.392+0000 7f5c61b0e980 20 INFO: main: starting shard=4 2024-03-04T17:45:22.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.392+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:22.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.392+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.392+0000 7f5c61b0e980 20 INFO: main: starting shard=5 2024-03-04T17:45:22.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.392+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:22.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.392+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-03-04T17:45:22.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.392+0000 7f5c61b0e980 20 INFO: main: starting shard=6 2024-03-04T17:45:22.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.392+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:22.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 INFO: main: starting shard=7 2024-03-04T17:45:22.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:22.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 INFO: main: starting shard=8 2024-03-04T17:45:22.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:22.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 INFO: main: starting shard=9 2024-03-04T17:45:22.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:22.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 INFO: main: starting shard=10 2024-03-04T17:45:22.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.396+0000 7f5c61b0e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:22.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.400+0000 7f5c61b0e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.400+0000 7f5c61b0e980 20 INFO: main: done 2024-03-04T17:45:22.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.400+0000 7f5b15fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.404+0000 7f5c61b0e980 20 remove_watcher() i=0 2024-03-04T17:45:22.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.404+0000 7f5c61b0e980 2 removed watcher, disabling cache 2024-03-04T17:45:22.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.408+0000 7f5c61b0e980 20 remove_watcher() i=1 2024-03-04T17:45:22.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.408+0000 7f5c61b0e980 20 remove_watcher() i=2 2024-03-04T17:45:22.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.412+0000 7f5c61b0e980 20 remove_watcher() i=3 2024-03-04T17:45:22.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.412+0000 7f5c61b0e980 20 remove_watcher() i=4 2024-03-04T17:45:22.420 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.416+0000 7f5c61b0e980 20 remove_watcher() i=5 2024-03-04T17:45:22.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.420+0000 7f5c61b0e980 20 remove_watcher() i=6 2024-03-04T17:45:22.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.424+0000 7f5c61b0e980 20 remove_watcher() i=7 2024-03-04T17:45:22.443 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: bucket check olh finds leftover OLH entries 2024-03-04T17:45:22.443 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:22.444 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-03-04T17:45:22.473 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:22.473 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:22.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.516+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 realm 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:22.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:22.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.520+0000 7f8932454980 20 searching for the correct realm 2024-03-04T17:45:22.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.528+0000 7f893076d640 0 -- 172.21.15.47:0/3683826723 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f892400e040 msgr2=0x7f892405aca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:22.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.536+0000 7f892ff6c640 0 -- 172.21.15.47:0/3683826723 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f8928121640 msgr2=0x7f892811c5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:22.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.540+0000 7f8932454980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:22.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.540+0000 7f8932454980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:22.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.540+0000 7f8932454980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:22.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.540+0000 7f8932454980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:22.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.540+0000 7f8932454980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:22.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.540+0000 7f8932454980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:22.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:22.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 zone default found 2024-03-04T17:45:22.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 4 Realm: () 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 zonegroup default 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 started sync module instance, tier type = 2024-03-04T17:45:22.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.544+0000 7f8932454980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:22.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.564+0000 7f8932454980 20 add_watcher() i=0 2024-03-04T17:45:22.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.564+0000 7f8932454980 20 add_watcher() i=1 2024-03-04T17:45:22.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.564+0000 7f8932454980 20 add_watcher() i=2 2024-03-04T17:45:22.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.564+0000 7f8932454980 20 add_watcher() i=3 2024-03-04T17:45:22.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.564+0000 7f8932454980 20 add_watcher() i=4 2024-03-04T17:45:22.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.564+0000 7f8932454980 20 add_watcher() i=5 2024-03-04T17:45:22.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.564+0000 7f8932454980 20 add_watcher() i=6 2024-03-04T17:45:22.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.568+0000 7f8932454980 20 add_watcher() i=7 2024-03-04T17:45:22.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.568+0000 7f8932454980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:22.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f87e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:22.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:22.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 5 note: GC not initialized 2024-03-04T17:45:22.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f87e4fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:22.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:22.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 20 Filter name: none 2024-03-04T17:45:22.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f875e7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:22.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:22.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:22.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:22.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:22.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:22.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:22.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:22.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:22.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:22.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:22.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.572+0000 7f8932454980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:22.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.576+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:22.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.576+0000 7f8932454980 1 NOTICE: finished shard 0 (0 entries found) 2024-03-04T17:45:22.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.576+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:22.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.576+0000 7f8932454980 1 NOTICE: finished shard 1 (0 entries found) 2024-03-04T17:45:22.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.576+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:22.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.576+0000 7f8932454980 1 NOTICE: finished shard 2 (0 entries found) 2024-03-04T17:45:22.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.576+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:22.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.576+0000 7f8932454980 1 NOTICE: finished shard 3 (0 entries found) 2024-03-04T17:45:22.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.576+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:22.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 1 NOTICE: finished shard 4 (0 entries found) 2024-03-04T17:45:22.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:22.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 1 NOTICE: finished shard 5 (1 entries found) 2024-03-04T17:45:22.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:22.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 1 NOTICE: finished shard 6 (0 entries found) 2024-03-04T17:45:22.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:22.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 1 NOTICE: finished shard 7 (0 entries found) 2024-03-04T17:45:22.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:22.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 1 NOTICE: finished shard 8 (1 entries found) 2024-03-04T17:45:22.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.580+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:22.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.584+0000 7f8932454980 1 NOTICE: finished shard 9 (0 entries found) 2024-03-04T17:45:22.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.584+0000 7f8932454980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:22.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.584+0000 7f8932454980 1 NOTICE: finished shard 10 (0 entries found) 2024-03-04T17:45:22.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.584+0000 7f8932454980 1 NOTICE: finished all shards (2 entries found) 2024-03-04T17:45:22.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.584+0000 7f87e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.588+0000 7f8932454980 20 remove_watcher() i=0 2024-03-04T17:45:22.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.588+0000 7f8932454980 2 removed watcher, disabling cache 2024-03-04T17:45:22.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.592+0000 7f8932454980 20 remove_watcher() i=1 2024-03-04T17:45:22.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.596+0000 7f8932454980 20 remove_watcher() i=2 2024-03-04T17:45:22.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.596+0000 7f8932454980 20 remove_watcher() i=3 2024-03-04T17:45:22.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.600+0000 7f8932454980 20 remove_watcher() i=4 2024-03-04T17:45:22.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.600+0000 7f8932454980 20 remove_watcher() i=5 2024-03-04T17:45:22.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.604+0000 7f8932454980 20 remove_watcher() i=6 2024-03-04T17:45:22.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.608+0000 7f8932454980 20 remove_watcher() i=7 2024-03-04T17:45:22.626 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: bucket check olh fixes leftover OLH entries 2024-03-04T17:45:22.626 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:22.627 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check olh --bucket check-bucket --fix --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-03-04T17:45:22.664 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:22.665 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:22.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.704+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.704+0000 7f072bf68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.704+0000 7f072bf68980 20 realm 2024-03-04T17:45:22.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.704+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.704+0000 7f072bf68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.704+0000 7f072bf68980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:22.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.704+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.704+0000 7f072bf68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.704+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.708+0000 7f072bf68980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.708+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.708+0000 7f072bf68980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:22.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.708+0000 7f072bf68980 20 searching for the correct realm 2024-03-04T17:45:22.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.716+0000 7f072a281640 0 -- 172.21.15.47:0/2362437807 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55b768b9be50 msgr2=0x55b768b9e310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:22.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.720+0000 7f072927f640 0 -- 172.21.15.47:0/2362437807 >> [v2:172.21.15.152:6808/929675582,v1:172.21.15.152:6809/929675582] conn(0x55b768ba52d0 msgr2=0x55b768ba7770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:22.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.728+0000 7f072bf68980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:22.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.728+0000 7f072bf68980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:22.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.728+0000 7f072bf68980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:22.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.728+0000 7f072bf68980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:22.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.728+0000 7f072bf68980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:22.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.728+0000 7f072bf68980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:22.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.728+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:22.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 zone default found 2024-03-04T17:45:22.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 4 Realm: () 2024-03-04T17:45:22.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:22.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:22.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:22.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 zonegroup default 2024-03-04T17:45:22.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 started sync module instance, tier type = 2024-03-04T17:45:22.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.732+0000 7f072bf68980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:22.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.756+0000 7f072bf68980 20 add_watcher() i=0 2024-03-04T17:45:22.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.756+0000 7f072bf68980 20 add_watcher() i=1 2024-03-04T17:45:22.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.756+0000 7f072bf68980 20 add_watcher() i=2 2024-03-04T17:45:22.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.756+0000 7f072bf68980 20 add_watcher() i=3 2024-03-04T17:45:22.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.756+0000 7f072bf68980 20 add_watcher() i=4 2024-03-04T17:45:22.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.756+0000 7f072bf68980 20 add_watcher() i=5 2024-03-04T17:45:22.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.756+0000 7f072bf68980 20 add_watcher() i=6 2024-03-04T17:45:22.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.756+0000 7f072bf68980 20 add_watcher() i=7 2024-03-04T17:45:22.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.756+0000 7f072bf68980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:22.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f05db7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f072bf68980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:22.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f072bf68980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:22.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f072bf68980 5 note: GC not initialized 2024-03-04T17:45:22.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f05da7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:22.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f072bf68980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:22.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f072bf68980 20 Filter name: none 2024-03-04T17:45:22.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f072bf68980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:22.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.760+0000 7f054bfff640 20 reqs_thread_entry: start 2024-03-04T17:45:22.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:22.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:22.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:22.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:22.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:22.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:22.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:22.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:22.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:22.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:22.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:22.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 1 NOTICE: finished shard 0 (0 entries removed) 2024-03-04T17:45:22.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:22.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 1 NOTICE: finished shard 1 (0 entries removed) 2024-03-04T17:45:22.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:22.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 1 NOTICE: finished shard 2 (0 entries removed) 2024-03-04T17:45:22.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.764+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:22.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.768+0000 7f072bf68980 1 NOTICE: finished shard 3 (0 entries removed) 2024-03-04T17:45:22.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.768+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:22.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.768+0000 7f072bf68980 1 NOTICE: finished shard 4 (0 entries removed) 2024-03-04T17:45:22.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.768+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:22.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.768+0000 7f072bf68980 20 get_obj_state: octx=0x55b76892e940 obj=check-bucket:d state=0x55b768bda238 s->prefetch_data=0 2024-03-04T17:45:22.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.768+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:22.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.768+0000 7f072bf68980 1 NOTICE: finished shard 6 (0 entries removed) 2024-03-04T17:45:22.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.768+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:22.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 1 NOTICE: finished shard 7 (0 entries removed) 2024-03-04T17:45:22.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:22.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 get_obj_state: octx=0x55b76893fbe0 obj=check-bucket:c state=0x55b768bcfe38 s->prefetch_data=0 2024-03-04T17:45:22.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:22.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 1 NOTICE: finished shard 9 (0 entries removed) 2024-03-04T17:45:22.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:22.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 1 NOTICE: finished shard 10 (0 entries removed) 2024-03-04T17:45:22.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 get_obj_state: setting s->obj_tag to ixyr3rcvls8k0m4ea753mvuv3boh21fy 2024-03-04T17:45:22.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 get_obj_state_impl: setting s->olh_tag to 2iygikqtxg0fyltrw72jf6cnysli3jzg 2024-03-04T17:45:22.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:22.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 get_obj_state: setting s->obj_tag to ncllb1gi35iifd5o18jy11ozauzav507 2024-03-04T17:45:22.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 get_obj_state_impl: setting s->olh_tag to s9y11ad4sqn1tx7eoth4dsodgkgdhkzs 2024-03-04T17:45:22.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:22.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.772+0000 7f072bf68980 20 olh_log_entry: epoch=4 op=2 key=c[] 2024-03-04T17:45:22.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.776+0000 7f072bf68980 20 olh_log_entry: epoch=4 op=3 key=c[hAJQdeQcyl0OCCOEJbIbxbpszbW-7F5] 2024-03-04T17:45:22.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.776+0000 7f072bf68980 20 get_obj_state: octx=0x55b768c22db0 obj=check-bucket:_:hAJQdeQcyl0OCCOEJbIbxbpszbW-7F5_c state=0x55b768c285b8 s->prefetch_data=0 2024-03-04T17:45:22.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.776+0000 7f072bf68980 20 olh_log_entry: epoch=4 op=2 key=d[] 2024-03-04T17:45:22.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.776+0000 7f072bf68980 20 olh_log_entry: epoch=4 op=3 key=d[mVdaP4ZoF6OZbDl7EK616h3EFZVHyqj] 2024-03-04T17:45:22.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.776+0000 7f072bf68980 20 get_obj_state: octx=0x55b768c02db0 obj=check-bucket:_:mVdaP4ZoF6OZbDl7EK616h3EFZVHyqj_d state=0x55b768c2e168 s->prefetch_data=0 2024-03-04T17:45:22.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.776+0000 7f072bf68980 20 get_obj_state: octx=0x55b768c22db0 obj=check-bucket:c state=0x55b768c23568 s->prefetch_data=0 2024-03-04T17:45:22.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.780+0000 7f072bf68980 20 get_obj_state: setting s->obj_tag to ncllb1gi35iifd5o18jy11ozauzav507 2024-03-04T17:45:22.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.780+0000 7f072bf68980 20 get_obj_state_impl: setting s->olh_tag to s9y11ad4sqn1tx7eoth4dsodgkgdhkzs 2024-03-04T17:45:22.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.780+0000 7f072bf68980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=s9y11ad4sqn1tx7eoth4dsodgkgdhkzs 2024-03-04T17:45:22.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.780+0000 7f072bf68980 20 get_obj_state: octx=0x55b768c02db0 obj=check-bucket:d state=0x55b768c28a38 s->prefetch_data=0 2024-03-04T17:45:22.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.780+0000 7f072bf68980 20 get_obj_state: setting s->obj_tag to ixyr3rcvls8k0m4ea753mvuv3boh21fy 2024-03-04T17:45:22.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.780+0000 7f072bf68980 20 get_obj_state_impl: setting s->olh_tag to 2iygikqtxg0fyltrw72jf6cnysli3jzg 2024-03-04T17:45:22.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.780+0000 7f072bf68980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=2iygikqtxg0fyltrw72jf6cnysli3jzg 2024-03-04T17:45:22.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.788+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:22.793 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.788+0000 7f072bf68980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:22.793 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.792+0000 7f072bf68980 1 NOTICE: finished shard 8 (1 entries removed) 2024-03-04T17:45:22.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.792+0000 7f072bf68980 1 NOTICE: finished shard 5 (1 entries removed) 2024-03-04T17:45:22.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.792+0000 7f072bf68980 1 NOTICE: finished all shards (2 entries removed) 2024-03-04T17:45:22.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.796+0000 7f05db7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.800+0000 7f072bf68980 20 remove_watcher() i=0 2024-03-04T17:45:22.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.800+0000 7f072bf68980 2 removed watcher, disabling cache 2024-03-04T17:45:22.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.800+0000 7f072bf68980 20 remove_watcher() i=1 2024-03-04T17:45:22.806 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.804+0000 7f072bf68980 20 remove_watcher() i=2 2024-03-04T17:45:22.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.804+0000 7f072bf68980 20 remove_watcher() i=3 2024-03-04T17:45:22.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.808+0000 7f072bf68980 20 remove_watcher() i=4 2024-03-04T17:45:22.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.808+0000 7f072bf68980 20 remove_watcher() i=5 2024-03-04T17:45:22.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.812+0000 7f072bf68980 20 remove_watcher() i=6 2024-03-04T17:45:22.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.816+0000 7f072bf68980 20 remove_watcher() i=7 2024-03-04T17:45:22.833 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-03-04T17:45:22.862 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:22.862 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:22.905 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.900+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 20 realm 2024-03-04T17:45:22.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:22.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.904+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.908+0000 7f27b37aa980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:22.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.908+0000 7f27b37aa980 20 searching for the correct realm 2024-03-04T17:45:22.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.912+0000 7f27b1ac3640 0 -- 172.21.15.47:0/2224757629 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f27a80e5940 msgr2=0x7f27a80ad450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:22.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.920+0000 7f27b12c2640 0 -- 172.21.15.47:0/2224757629 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f27ac022040 msgr2=0x7f27ac030410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:22.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.924+0000 7f27b37aa980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:22.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.924+0000 7f27b37aa980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:22.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.924+0000 7f27b37aa980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:22.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.924+0000 7f27b37aa980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:22.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.924+0000 7f27b37aa980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:22.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.924+0000 7f27b37aa980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:22.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.924+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 zone default found 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 4 Realm: () 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 zonegroup default 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 started sync module instance, tier type = 2024-03-04T17:45:22.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.928+0000 7f27b37aa980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:22.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.952+0000 7f27b37aa980 20 add_watcher() i=0 2024-03-04T17:45:22.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.952+0000 7f27b37aa980 20 add_watcher() i=1 2024-03-04T17:45:22.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.952+0000 7f27b37aa980 20 add_watcher() i=2 2024-03-04T17:45:22.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.952+0000 7f27b37aa980 20 add_watcher() i=3 2024-03-04T17:45:22.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.952+0000 7f27b37aa980 20 add_watcher() i=4 2024-03-04T17:45:22.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.952+0000 7f27b37aa980 20 add_watcher() i=5 2024-03-04T17:45:22.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.952+0000 7f27b37aa980 20 add_watcher() i=6 2024-03-04T17:45:22.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.956+0000 7f27b37aa980 20 add_watcher() i=7 2024-03-04T17:45:22.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.956+0000 7f27b37aa980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:22.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f2667fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f27b37aa980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:22.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f27b37aa980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:22.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f27b37aa980 5 note: GC not initialized 2024-03-04T17:45:22.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f2666fe5640 20 reqs_thread_entry: start 2024-03-04T17:45:22.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f27b37aa980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:22.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f27b37aa980 20 Filter name: none 2024-03-04T17:45:22.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f2664fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:22.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f27b37aa980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:22.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.960+0000 7f27b37aa980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:22.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:22.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:22.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: starting shard=0 2024-03-04T17:45:22.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:22.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: starting shard=1 2024-03-04T17:45:22.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:22.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: starting shard=2 2024-03-04T17:45:22.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:22.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: starting shard=3 2024-03-04T17:45:22.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:22.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 INFO: main: starting shard=4 2024-03-04T17:45:22.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.964+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:22.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 INFO: main: starting shard=5 2024-03-04T17:45:22.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:22.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 INFO: main: starting shard=6 2024-03-04T17:45:22.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:22.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 INFO: main: starting shard=7 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 INFO: main: starting shard=8 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.968+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.972+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.972+0000 7f27b37aa980 20 INFO: main: starting shard=9 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.972+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.972+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.972+0000 7f27b37aa980 20 INFO: main: starting shard=10 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.972+0000 7f27b37aa980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.972+0000 7f27b37aa980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.972+0000 7f27b37aa980 20 INFO: main: done 2024-03-04T17:45:22.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.976+0000 7f2667fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.982 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.980+0000 7f27b37aa980 20 remove_watcher() i=0 2024-03-04T17:45:22.982 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.980+0000 7f27b37aa980 2 removed watcher, disabling cache 2024-03-04T17:45:22.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.980+0000 7f27b37aa980 20 remove_watcher() i=1 2024-03-04T17:45:22.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.984+0000 7f27b37aa980 20 remove_watcher() i=2 2024-03-04T17:45:22.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.984+0000 7f27b37aa980 20 remove_watcher() i=3 2024-03-04T17:45:22.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.988+0000 7f27b37aa980 20 remove_watcher() i=4 2024-03-04T17:45:22.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.992+0000 7f27b37aa980 20 remove_watcher() i=5 2024-03-04T17:45:22.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.992+0000 7f27b37aa980 20 remove_watcher() i=6 2024-03-04T17:45:22.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:22.996+0000 7f27b37aa980 20 remove_watcher() i=7 2024-03-04T17:45:23.014 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-03-04T17:45:23.043 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:23.043 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:23.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.084+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.084+0000 7f9c3e987980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.084+0000 7f9c3e987980 20 realm 2024-03-04T17:45:23.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.084+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.084+0000 7f9c3e987980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.084+0000 7f9c3e987980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:23.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.084+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.088+0000 7f9c3e987980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.088+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.088+0000 7f9c3e987980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:23.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.088+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.088+0000 7f9c3e987980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:23.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.088+0000 7f9c3e987980 20 searching for the correct realm 2024-03-04T17:45:23.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.096+0000 7f9c3cca0640 0 -- 172.21.15.47:0/1976394721 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f9c300c2950 msgr2=0x7f9c300e8410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:23.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.100+0000 7f9c3c49f640 0 -- 172.21.15.47:0/1976394721 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f9c34058180 msgr2=0x7f9c34001ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:23.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:23.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:23.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:23.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:23.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:23.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:23.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:23.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:23.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 zone default found 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 4 Realm: () 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 zonegroup default 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.108+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.112+0000 7f9c3e987980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.112+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.112+0000 7f9c3e987980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.112+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.112+0000 7f9c3e987980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.112+0000 7f9c3e987980 20 started sync module instance, tier type = 2024-03-04T17:45:23.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.112+0000 7f9c3e987980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:23.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.132+0000 7f9c3e987980 20 add_watcher() i=0 2024-03-04T17:45:23.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.132+0000 7f9c3e987980 20 add_watcher() i=1 2024-03-04T17:45:23.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.132+0000 7f9c3e987980 20 add_watcher() i=2 2024-03-04T17:45:23.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.132+0000 7f9c3e987980 20 add_watcher() i=3 2024-03-04T17:45:23.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.132+0000 7f9c3e987980 20 add_watcher() i=4 2024-03-04T17:45:23.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.132+0000 7f9c3e987980 20 add_watcher() i=5 2024-03-04T17:45:23.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.132+0000 7f9c3e987980 20 add_watcher() i=6 2024-03-04T17:45:23.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.132+0000 7f9c3e987980 20 add_watcher() i=7 2024-03-04T17:45:23.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.132+0000 7f9c3e987980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:23.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.136+0000 7f9af2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:23.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.136+0000 7f9c3e987980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:23.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.136+0000 7f9c3e987980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:23.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.136+0000 7f9c3e987980 5 note: GC not initialized 2024-03-04T17:45:23.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.136+0000 7f9af1fdb640 20 reqs_thread_entry: start 2024-03-04T17:45:23.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.136+0000 7f9c3e987980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:23.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.136+0000 7f9c3e987980 20 Filter name: none 2024-03-04T17:45:23.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.136+0000 7f9a637fe640 20 reqs_thread_entry: start 2024-03-04T17:45:23.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.136+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:23.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:23.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:23.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:23.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 INFO: main: starting shard=0 2024-03-04T17:45:23.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:23.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 INFO: main: starting shard=1 2024-03-04T17:45:23.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:23.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 INFO: main: starting shard=2 2024-03-04T17:45:23.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.140+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:23.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: starting shard=3 2024-03-04T17:45:23.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:23.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: starting shard=4 2024-03-04T17:45:23.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:23.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: starting shard=5 2024-03-04T17:45:23.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:23.148 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.148 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: starting shard=6 2024-03-04T17:45:23.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:23.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 INFO: main: starting shard=7 2024-03-04T17:45:23.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.144+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:23.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 INFO: main: starting shard=8 2024-03-04T17:45:23.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:23.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 INFO: main: starting shard=9 2024-03-04T17:45:23.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:23.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 INFO: main: starting shard=10 2024-03-04T17:45:23.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:23.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.148+0000 7f9c3e987980 20 INFO: main: done 2024-03-04T17:45:23.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.152+0000 7f9af2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:23.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.156+0000 7f9c3e987980 20 remove_watcher() i=0 2024-03-04T17:45:23.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.156+0000 7f9c3e987980 2 removed watcher, disabling cache 2024-03-04T17:45:23.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.156+0000 7f9c3e987980 20 remove_watcher() i=1 2024-03-04T17:45:23.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.160+0000 7f9c3e987980 20 remove_watcher() i=2 2024-03-04T17:45:23.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.160+0000 7f9c3e987980 20 remove_watcher() i=3 2024-03-04T17:45:23.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.164+0000 7f9c3e987980 20 remove_watcher() i=4 2024-03-04T17:45:23.168 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.164+0000 7f9c3e987980 20 remove_watcher() i=5 2024-03-04T17:45:23.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.168+0000 7f9c3e987980 20 remove_watcher() i=6 2024-03-04T17:45:23.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.172+0000 7f9c3e987980 20 remove_watcher() i=7 2024-03-04T17:45:23.189 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: bucket check olh does not find new leftover OLH entries after fix 2024-03-04T17:45:23.189 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:23.189 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-03-04T17:45:23.218 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:23.218 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:23.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.256+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.260+0000 7f1a730fd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.260+0000 7f1a730fd980 20 realm 2024-03-04T17:45:23.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.260+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.260+0000 7f1a730fd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.260+0000 7f1a730fd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:23.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.260+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.260+0000 7f1a730fd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.260+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.264+0000 7f1a730fd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:23.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.264+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.264+0000 7f1a730fd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:23.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.264+0000 7f1a730fd980 20 searching for the correct realm 2024-03-04T17:45:23.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.268+0000 7f1a70414640 0 -- 172.21.15.47:0/1373108855 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f1a640d0450 msgr2=0x7f1a640d08d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:23.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.276+0000 7f1a71416640 0 -- 172.21.15.47:0/1373108855 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x5613673779c0 msgr2=0x561367379e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:23.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.284+0000 7f1a730fd980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:23.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.284+0000 7f1a730fd980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:23.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.284+0000 7f1a730fd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:23.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.284+0000 7f1a730fd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:23.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.284+0000 7f1a730fd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:23.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.284+0000 7f1a730fd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.284+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:23.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 zone default found 2024-03-04T17:45:23.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 4 Realm: () 2024-03-04T17:45:23.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:23.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:23.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:23.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 zonegroup default 2024-03-04T17:45:23.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 started sync module instance, tier type = 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.288+0000 7f1a730fd980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:23.310 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.308+0000 7f1a730fd980 20 add_watcher() i=0 2024-03-04T17:45:23.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.308+0000 7f1a730fd980 20 add_watcher() i=1 2024-03-04T17:45:23.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.308+0000 7f1a730fd980 20 add_watcher() i=2 2024-03-04T17:45:23.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.308+0000 7f1a730fd980 20 add_watcher() i=3 2024-03-04T17:45:23.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.308+0000 7f1a730fd980 20 add_watcher() i=4 2024-03-04T17:45:23.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.308+0000 7f1a730fd980 20 add_watcher() i=5 2024-03-04T17:45:23.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.308+0000 7f1a730fd980 20 add_watcher() i=6 2024-03-04T17:45:23.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.308+0000 7f1a730fd980 20 add_watcher() i=7 2024-03-04T17:45:23.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.308+0000 7f1a730fd980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:23.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.312+0000 7f1926fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:23.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.312+0000 7f1a730fd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:23.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.312+0000 7f1a730fd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:23.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.312+0000 7f1a730fd980 5 note: GC not initialized 2024-03-04T17:45:23.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1925fe3640 20 reqs_thread_entry: start 2024-03-04T17:45:23.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:23.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 20 Filter name: none 2024-03-04T17:45:23.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f18977fe640 20 reqs_thread_entry: start 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:23.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:23.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:23.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:23.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:23.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:23.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:23.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 1 NOTICE: finished shard 0 (0 entries found) 2024-03-04T17:45:23.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.316+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:23.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 1 NOTICE: finished shard 1 (0 entries found) 2024-03-04T17:45:23.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:23.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 1 NOTICE: finished shard 2 (0 entries found) 2024-03-04T17:45:23.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:23.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 1 NOTICE: finished shard 3 (0 entries found) 2024-03-04T17:45:23.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:23.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 1 NOTICE: finished shard 4 (0 entries found) 2024-03-04T17:45:23.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:23.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 1 NOTICE: finished shard 5 (0 entries found) 2024-03-04T17:45:23.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.320+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:23.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 1 NOTICE: finished shard 6 (0 entries found) 2024-03-04T17:45:23.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:23.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 1 NOTICE: finished shard 7 (0 entries found) 2024-03-04T17:45:23.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:23.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 1 NOTICE: finished shard 8 (0 entries found) 2024-03-04T17:45:23.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:23.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 1 NOTICE: finished shard 9 (0 entries found) 2024-03-04T17:45:23.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:23.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 1 NOTICE: finished shard 10 (0 entries found) 2024-03-04T17:45:23.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.324+0000 7f1a730fd980 1 NOTICE: finished all shards (0 entries found) 2024-03-04T17:45:23.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.328+0000 7f1926fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:23.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.332+0000 7f1a730fd980 20 remove_watcher() i=0 2024-03-04T17:45:23.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.332+0000 7f1a730fd980 2 removed watcher, disabling cache 2024-03-04T17:45:23.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.332+0000 7f1a730fd980 20 remove_watcher() i=1 2024-03-04T17:45:23.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.336+0000 7f1a730fd980 20 remove_watcher() i=2 2024-03-04T17:45:23.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.336+0000 7f1a730fd980 20 remove_watcher() i=3 2024-03-04T17:45:23.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.340+0000 7f1a730fd980 20 remove_watcher() i=4 2024-03-04T17:45:23.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.340+0000 7f1a730fd980 20 remove_watcher() i=5 2024-03-04T17:45:23.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.344+0000 7f1a730fd980 20 remove_watcher() i=6 2024-03-04T17:45:23.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:23.348+0000 7f1a730fd980 20 remove_watcher() i=7 2024-03-04T17:45:23.365 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: verify that bucket check fixes do not affect null version objects 2024-03-04T17:45:23.365 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:23.366 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:23.367 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:23.392 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:23.392 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:23.419 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:23.419 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:23.510 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: bucket check stats are correct in the presence of unlinked entries 2024-03-04T17:45:23.510 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:45:23.512 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:23.512 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:23.630 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:23.630 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:24.092 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.095 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:24.244 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.244 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:24.309 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.309 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:24.444 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.444 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:24.509 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.509 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:24.641 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.642 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:24.707 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.707 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:24.842 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.842 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:24.909 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.909 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:25.047 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.047 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:25.113 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.113 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:25.234 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.234 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:25.298 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-03-04T17:45:25.732 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-03-04T17:45:27.172 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:27.172 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:27.307 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:27.307 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-03-04T17:45:27.343 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:27.343 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:27.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.384+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.384+0000 7fbb38377980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.384+0000 7fbb38377980 20 realm 2024-03-04T17:45:27.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.384+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.388+0000 7fbb38377980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.388+0000 7fbb38377980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:27.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.388+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.388+0000 7fbb38377980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.388+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.388+0000 7fbb38377980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:27.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.388+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.388+0000 7fbb38377980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:27.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.388+0000 7fbb38377980 20 searching for the correct realm 2024-03-04T17:45:27.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.396+0000 7fbb35e8f640 0 -- 172.21.15.47:0/2666414684 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fbb280d8030 msgr2=0x7fbb28002dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:27.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.404+0000 7fbb35e8f640 0 -- 172.21.15.47:0/2666414684 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fbb30058d20 msgr2=0x7fbb30057980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:27.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:27.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:27.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:27.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:27.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:27.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:27.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.414 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.408+0000 7fbb38377980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 zone default found 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 4 Realm: () 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 zonegroup default 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 started sync module instance, tier type = 2024-03-04T17:45:27.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.412+0000 7fbb38377980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:27.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.432+0000 7fbb38377980 20 add_watcher() i=0 2024-03-04T17:45:27.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.432+0000 7fbb38377980 20 add_watcher() i=1 2024-03-04T17:45:27.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.432+0000 7fbb38377980 20 add_watcher() i=2 2024-03-04T17:45:27.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.432+0000 7fbb38377980 20 add_watcher() i=3 2024-03-04T17:45:27.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.432+0000 7fbb38377980 20 add_watcher() i=4 2024-03-04T17:45:27.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.432+0000 7fbb38377980 20 add_watcher() i=5 2024-03-04T17:45:27.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.432+0000 7fbb38377980 20 add_watcher() i=6 2024-03-04T17:45:27.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.432+0000 7fbb38377980 20 add_watcher() i=7 2024-03-04T17:45:27.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.432+0000 7fbb38377980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:27.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.436+0000 7fb9e77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:27.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.436+0000 7fbb38377980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:27.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.436+0000 7fbb38377980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:27.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.436+0000 7fbb38377980 5 note: GC not initialized 2024-03-04T17:45:27.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fb9e67fc640 20 reqs_thread_entry: start 2024-03-04T17:45:27.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:27.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 Filter name: none 2024-03-04T17:45:27.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fb957fff640 20 reqs_thread_entry: start 2024-03-04T17:45:27.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:27.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:27.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:27.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:27.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 INFO: main: starting shard=0 2024-03-04T17:45:27.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:27.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 INFO: main: starting shard=1 2024-03-04T17:45:27.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.440+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:27.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: starting shard=2 2024-03-04T17:45:27.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:27.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: starting shard=3 2024-03-04T17:45:27.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:27.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: starting shard=4 2024-03-04T17:45:27.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:27.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: starting shard=5 2024-03-04T17:45:27.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:27.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 INFO: main: starting shard=6 2024-03-04T17:45:27.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.444+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:27.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 INFO: main: starting shard=7 2024-03-04T17:45:27.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:27.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 INFO: main: starting shard=8 2024-03-04T17:45:27.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:27.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-03-04T17:45:27.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 INFO: main: starting shard=9 2024-03-04T17:45:27.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:27.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 INFO: main: starting shard=10 2024-03-04T17:45:27.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.448+0000 7fbb38377980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:27.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.452+0000 7fbb38377980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:27.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.452+0000 7fbb38377980 20 INFO: main: done 2024-03-04T17:45:27.455 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.452+0000 7fb9e77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:27.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.456+0000 7fbb38377980 20 remove_watcher() i=0 2024-03-04T17:45:27.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.456+0000 7fbb38377980 2 removed watcher, disabling cache 2024-03-04T17:45:27.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.460+0000 7fbb38377980 20 remove_watcher() i=1 2024-03-04T17:45:27.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.460+0000 7fbb38377980 20 remove_watcher() i=2 2024-03-04T17:45:27.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.464+0000 7fbb38377980 20 remove_watcher() i=3 2024-03-04T17:45:27.469 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.464+0000 7fbb38377980 20 remove_watcher() i=4 2024-03-04T17:45:27.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.468+0000 7fbb38377980 20 remove_watcher() i=5 2024-03-04T17:45:27.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.472+0000 7fbb38377980 20 remove_watcher() i=6 2024-03-04T17:45:27.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.472+0000 7fbb38377980 20 remove_watcher() i=7 2024-03-04T17:45:27.494 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:27.494 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:27.518 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:27.518 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:27.623 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:27.623 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-03-04T17:45:27.662 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:27.663 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:27.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.704+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 realm 2024-03-04T17:45:27.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:27.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:27.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:27.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.708+0000 7f1627150980 20 searching for the correct realm 2024-03-04T17:45:27.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.716+0000 7f1624c68640 0 -- 172.21.15.47:0/388822135 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f1614122c40 msgr2=0x7f1614122070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:27.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:27.928+0000 7f1624c68640 0 -- 172.21.15.47:0/388822135 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x55d5a0f4f070 msgr2=0x7f161c019420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:28.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:28.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:28.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:28.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:28.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:28.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:28.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:28.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:28.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 zone default found 2024-03-04T17:45:28.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 4 Realm: () 2024-03-04T17:45:28.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:28.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:28.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:28.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 zonegroup default 2024-03-04T17:45:28.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.148+0000 7f1627150980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.152+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.152+0000 7f1627150980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.152+0000 7f1627150980 20 started sync module instance, tier type = 2024-03-04T17:45:28.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.152+0000 7f1627150980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:28.181 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.176+0000 7f1627150980 20 add_watcher() i=0 2024-03-04T17:45:28.181 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.176+0000 7f1627150980 20 add_watcher() i=1 2024-03-04T17:45:28.181 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.176+0000 7f1627150980 20 add_watcher() i=2 2024-03-04T17:45:28.181 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.176+0000 7f1627150980 20 add_watcher() i=3 2024-03-04T17:45:28.181 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.176+0000 7f1627150980 20 add_watcher() i=4 2024-03-04T17:45:28.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.176+0000 7f1627150980 20 add_watcher() i=5 2024-03-04T17:45:28.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.180+0000 7f1627150980 20 add_watcher() i=6 2024-03-04T17:45:28.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.184+0000 7f1627150980 20 add_watcher() i=7 2024-03-04T17:45:28.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.184+0000 7f1627150980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:28.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.188+0000 7f14dafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:28.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.188+0000 7f1627150980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:28.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.188+0000 7f1627150980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:28.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.188+0000 7f1627150980 5 note: GC not initialized 2024-03-04T17:45:28.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.188+0000 7f14d9fe3640 20 reqs_thread_entry: start 2024-03-04T17:45:28.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.188+0000 7f1627150980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:28.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.188+0000 7f1627150980 20 Filter name: none 2024-03-04T17:45:28.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.188+0000 7f144b7fe640 20 reqs_thread_entry: start 2024-03-04T17:45:28.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:28.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:28.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:28.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:28.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 INFO: main: starting shard=0 2024-03-04T17:45:28.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:28.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 INFO: main: starting shard=1 2024-03-04T17:45:28.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.192+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:28.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 INFO: main: starting shard=2 2024-03-04T17:45:28.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:28.199 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.199 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 INFO: main: starting shard=3 2024-03-04T17:45:28.199 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:28.199 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 INFO: main: starting shard=4 2024-03-04T17:45:28.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:28.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 INFO: main: starting shard=5 2024-03-04T17:45:28.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.196+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:28.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-03-04T17:45:28.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 INFO: main: starting shard=6 2024-03-04T17:45:28.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:28.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 INFO: main: starting shard=7 2024-03-04T17:45:28.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:28.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 INFO: main: starting shard=8 2024-03-04T17:45:28.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:28.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 INFO: main: starting shard=9 2024-03-04T17:45:28.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.200+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:28.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.204+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.204+0000 7f1627150980 20 INFO: main: starting shard=10 2024-03-04T17:45:28.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.204+0000 7f1627150980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:28.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.204+0000 7f1627150980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.204+0000 7f1627150980 20 INFO: main: done 2024-03-04T17:45:28.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.208+0000 7f14dafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:28.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.212+0000 7f1627150980 20 remove_watcher() i=0 2024-03-04T17:45:28.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.212+0000 7f1627150980 2 removed watcher, disabling cache 2024-03-04T17:45:28.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.212+0000 7f1627150980 20 remove_watcher() i=1 2024-03-04T17:45:28.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.216+0000 7f1627150980 20 remove_watcher() i=2 2024-03-04T17:45:28.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.220+0000 7f1627150980 20 remove_watcher() i=3 2024-03-04T17:45:28.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.220+0000 7f1627150980 20 remove_watcher() i=4 2024-03-04T17:45:28.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.224+0000 7f1627150980 20 remove_watcher() i=5 2024-03-04T17:45:28.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.228+0000 7f1627150980 20 remove_watcher() i=6 2024-03-04T17:45:28.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.228+0000 7f1627150980 20 remove_watcher() i=7 2024-03-04T17:45:28.251 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.252 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:28.278 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.278 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:28.310 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.310 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:28.451 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:28.451 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-03-04T17:45:28.487 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:28.487 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:28.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.528+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 realm 2024-03-04T17:45:28.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:28.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:28.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:28.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.532+0000 7fbc44104980 20 searching for the correct realm 2024-03-04T17:45:28.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.540+0000 7fbc4141b640 0 -- 172.21.15.47:0/3918342118 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fbc3803b3f0 msgr2=0x7fbc3803b850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:28.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.756+0000 7fbc4241d640 0 -- 172.21.15.47:0/3918342118 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55da9b3ae920 msgr2=0x55da9b3b0d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:28.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:28.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:28.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:28.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:28.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:28.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:28.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:28.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:28.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 zone default found 2024-03-04T17:45:28.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 4 Realm: () 2024-03-04T17:45:28.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:28.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:28.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:28.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 zonegroup default 2024-03-04T17:45:28.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.768+0000 7fbc44104980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.772+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.772+0000 7fbc44104980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.772+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.772+0000 7fbc44104980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.772+0000 7fbc44104980 20 started sync module instance, tier type = 2024-03-04T17:45:28.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.772+0000 7fbc44104980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:28.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.796+0000 7fbc44104980 20 add_watcher() i=0 2024-03-04T17:45:28.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.796+0000 7fbc44104980 20 add_watcher() i=1 2024-03-04T17:45:28.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.796+0000 7fbc44104980 20 add_watcher() i=2 2024-03-04T17:45:28.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.796+0000 7fbc44104980 20 add_watcher() i=3 2024-03-04T17:45:28.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.796+0000 7fbc44104980 20 add_watcher() i=4 2024-03-04T17:45:28.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.796+0000 7fbc44104980 20 add_watcher() i=5 2024-03-04T17:45:28.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.796+0000 7fbc44104980 20 add_watcher() i=6 2024-03-04T17:45:28.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.800+0000 7fbc44104980 20 add_watcher() i=7 2024-03-04T17:45:28.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.800+0000 7fbc44104980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:28.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.804+0000 7fbaf37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:28.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.804+0000 7fbc44104980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:28.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.804+0000 7fbc44104980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:28.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.804+0000 7fbc44104980 5 note: GC not initialized 2024-03-04T17:45:28.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.804+0000 7fbaf27fc640 20 reqs_thread_entry: start 2024-03-04T17:45:28.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.804+0000 7fbc44104980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:28.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fbc44104980 20 Filter name: none 2024-03-04T17:45:28.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fba63fff640 20 reqs_thread_entry: start 2024-03-04T17:45:28.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fbc44104980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:28.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fbc44104980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fbc44104980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:28.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fbc44104980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:28.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fbc44104980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:28.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fbc44104980 20 INFO: main: starting shard=0 2024-03-04T17:45:28.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.808+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:28.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: starting shard=1 2024-03-04T17:45:28.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:28.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T17:45:28.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: starting shard=2 2024-03-04T17:45:28.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:28.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: starting shard=3 2024-03-04T17:45:28.816 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:28.816 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.816 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: starting shard=4 2024-03-04T17:45:28.816 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:28.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 INFO: main: starting shard=5 2024-03-04T17:45:28.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.812+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:28.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 INFO: main: starting shard=6 2024-03-04T17:45:28.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:28.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 INFO: main: starting shard=7 2024-03-04T17:45:28.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:28.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 INFO: main: starting shard=8 2024-03-04T17:45:28.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:28.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 INFO: main: starting shard=9 2024-03-04T17:45:28.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.816+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:28.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.820+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.820+0000 7fbc44104980 20 INFO: main: starting shard=10 2024-03-04T17:45:28.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.820+0000 7fbc44104980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:28.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.820+0000 7fbc44104980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:28.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.820+0000 7fbc44104980 20 INFO: main: done 2024-03-04T17:45:28.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.820+0000 7fbaf37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:28.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.824+0000 7fbc44104980 20 remove_watcher() i=0 2024-03-04T17:45:28.830 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.824+0000 7fbc44104980 2 removed watcher, disabling cache 2024-03-04T17:45:28.831 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.828+0000 7fbc44104980 20 remove_watcher() i=1 2024-03-04T17:45:28.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.832+0000 7fbc44104980 20 remove_watcher() i=2 2024-03-04T17:45:28.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.832+0000 7fbc44104980 20 remove_watcher() i=3 2024-03-04T17:45:28.838 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.836+0000 7fbc44104980 20 remove_watcher() i=4 2024-03-04T17:45:28.841 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.836+0000 7fbc44104980 20 remove_watcher() i=5 2024-03-04T17:45:28.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.840+0000 7fbc44104980 20 remove_watcher() i=6 2024-03-04T17:45:28.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:28.844+0000 7fbc44104980 20 remove_watcher() i=7 2024-03-04T17:45:28.866 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.866 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:28.890 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.890 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:28.995 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:28.995 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-03-04T17:45:29.031 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:29.031 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:29.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.072+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.072+0000 7f893d6a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.072+0000 7f893d6a4980 20 realm 2024-03-04T17:45:29.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.072+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.076+0000 7f893d6a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.076+0000 7f893d6a4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:29.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.076+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.076+0000 7f893d6a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.076+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.076+0000 7f893d6a4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:29.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.076+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.076+0000 7f893d6a4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:29.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.076+0000 7f893d6a4980 20 searching for the correct realm 2024-03-04T17:45:29.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.084+0000 7f893a9bb640 0 -- 172.21.15.47:0/3584403925 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f89300de410 msgr2=0x7f89300de890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:29.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.296+0000 7f893a9bb640 0 -- 172.21.15.47:0/3584403925 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x557cdf59e620 msgr2=0x557cdf5a0a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:29.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:29.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:29.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 zone default found 2024-03-04T17:45:29.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 4 Realm: () 2024-03-04T17:45:29.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:29.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:29.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:29.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 zonegroup default 2024-03-04T17:45:29.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.312+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.316+0000 7f893d6a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.316+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.316+0000 7f893d6a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.316+0000 7f893d6a4980 20 started sync module instance, tier type = 2024-03-04T17:45:29.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.316+0000 7f893d6a4980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:29.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.340+0000 7f893d6a4980 20 add_watcher() i=0 2024-03-04T17:45:29.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.340+0000 7f893d6a4980 20 add_watcher() i=1 2024-03-04T17:45:29.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.340+0000 7f893d6a4980 20 add_watcher() i=2 2024-03-04T17:45:29.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.340+0000 7f893d6a4980 20 add_watcher() i=3 2024-03-04T17:45:29.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.340+0000 7f893d6a4980 20 add_watcher() i=4 2024-03-04T17:45:29.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.340+0000 7f893d6a4980 20 add_watcher() i=5 2024-03-04T17:45:29.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.340+0000 7f893d6a4980 20 add_watcher() i=6 2024-03-04T17:45:29.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.344+0000 7f893d6a4980 20 add_watcher() i=7 2024-03-04T17:45:29.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.344+0000 7f893d6a4980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:29.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.348+0000 7f87f1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:29.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.348+0000 7f893d6a4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:29.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.348+0000 7f893d6a4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:29.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.348+0000 7f893d6a4980 5 note: GC not initialized 2024-03-04T17:45:29.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.348+0000 7f87f0fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:29.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.348+0000 7f893d6a4980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:29.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.348+0000 7f893d6a4980 20 Filter name: none 2024-03-04T17:45:29.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.348+0000 7f876a7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:29.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.348+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:29.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:29.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:29.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-03-04T17:45:29.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 INFO: main: starting shard=0 2024-03-04T17:45:29.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:29.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 INFO: main: starting shard=1 2024-03-04T17:45:29.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.352+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:29.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 INFO: main: starting shard=2 2024-03-04T17:45:29.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:29.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 INFO: main: starting shard=3 2024-03-04T17:45:29.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:29.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 INFO: main: starting shard=4 2024-03-04T17:45:29.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:29.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 INFO: main: starting shard=5 2024-03-04T17:45:29.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.356+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:29.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 INFO: main: starting shard=6 2024-03-04T17:45:29.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:29.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 INFO: main: starting shard=7 2024-03-04T17:45:29.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:29.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 INFO: main: starting shard=8 2024-03-04T17:45:29.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:29.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 INFO: main: starting shard=9 2024-03-04T17:45:29.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.360+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:29.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.364+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:29.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.364+0000 7f893d6a4980 20 INFO: main: starting shard=10 2024-03-04T17:45:29.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.364+0000 7f893d6a4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:29.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.364+0000 7f893d6a4980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T17:45:29.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.364+0000 7f893d6a4980 20 INFO: main: done 2024-03-04T17:45:29.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.364+0000 7f87f1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:29.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.372+0000 7f893d6a4980 20 remove_watcher() i=0 2024-03-04T17:45:29.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.372+0000 7f893d6a4980 2 removed watcher, disabling cache 2024-03-04T17:45:29.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.372+0000 7f893d6a4980 20 remove_watcher() i=1 2024-03-04T17:45:29.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.376+0000 7f893d6a4980 20 remove_watcher() i=2 2024-03-04T17:45:29.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.376+0000 7f893d6a4980 20 remove_watcher() i=3 2024-03-04T17:45:29.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.380+0000 7f893d6a4980 20 remove_watcher() i=4 2024-03-04T17:45:29.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.384+0000 7f893d6a4980 20 remove_watcher() i=5 2024-03-04T17:45:29.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.384+0000 7f893d6a4980 20 remove_watcher() i=6 2024-03-04T17:45:29.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:29.388+0000 7f893d6a4980 20 remove_watcher() i=7 2024-03-04T17:45:29.415 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:29.415 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:29.446 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-03-04T17:45:29.887 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-03-04T17:45:30.303 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-03-04T17:45:30.329 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:30.329 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:30.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.368+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.368+0000 7fee3e47d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.368+0000 7fee3e47d980 20 realm 2024-03-04T17:45:30.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.368+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.368+0000 7fee3e47d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.368+0000 7fee3e47d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:30.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.368+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.368+0000 7fee3e47d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.368+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.372+0000 7fee3e47d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:30.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.372+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.372+0000 7fee3e47d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:30.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.372+0000 7fee3e47d980 20 searching for the correct realm 2024-03-04T17:45:30.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.380+0000 7fee3c796640 0 -- 172.21.15.47:0/2065461138 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x559b1f409ed0 msgr2=0x559b1f40c340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:30.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.384+0000 7fee3bf95640 0 -- 172.21.15.47:0/2065461138 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x559b1f40e0f0 msgr2=0x559b1f410520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:30.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:30.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:30.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:30.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:30.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:30.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:30.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:30.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:30.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 zone default found 2024-03-04T17:45:30.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 4 Realm: () 2024-03-04T17:45:30.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:30.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:30.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:30.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 zonegroup default 2024-03-04T17:45:30.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.392+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.396+0000 7fee3e47d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.396+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.396+0000 7fee3e47d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.396+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.396+0000 7fee3e47d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.396+0000 7fee3e47d980 20 started sync module instance, tier type = 2024-03-04T17:45:30.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.396+0000 7fee3e47d980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:30.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.416+0000 7fee3e47d980 20 add_watcher() i=0 2024-03-04T17:45:30.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.416+0000 7fee3e47d980 20 add_watcher() i=1 2024-03-04T17:45:30.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.416+0000 7fee3e47d980 20 add_watcher() i=2 2024-03-04T17:45:30.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.416+0000 7fee3e47d980 20 add_watcher() i=3 2024-03-04T17:45:30.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.416+0000 7fee3e47d980 20 add_watcher() i=4 2024-03-04T17:45:30.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.416+0000 7fee3e47d980 20 add_watcher() i=5 2024-03-04T17:45:30.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.416+0000 7fee3e47d980 20 add_watcher() i=6 2024-03-04T17:45:30.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.416+0000 7fee3e47d980 20 add_watcher() i=7 2024-03-04T17:45:30.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.416+0000 7fee3e47d980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:30.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fecf1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:30.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fee3e47d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:30.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fee3e47d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:30.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fee3e47d980 5 note: GC not initialized 2024-03-04T17:45:30.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fecf0fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:30.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fee3e47d980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:30.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fee3e47d980 20 Filter name: none 2024-03-04T17:45:30.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fec6a7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:30.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fee3e47d980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:30.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.420+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:30.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:30.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:30.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:30.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:30.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:30.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:30.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:30.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:30.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:30.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 20 list_objects_ordered: starting attempt 1 2024-03-04T17:45:30.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T17:45:30.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T17:45:30.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T17:45:30.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T17:45:30.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.424+0000 7fee3e47d980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T17:45:30.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.436+0000 7fecf1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:30.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.440+0000 7fee3e47d980 20 remove_watcher() i=0 2024-03-04T17:45:30.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.440+0000 7fee3e47d980 2 removed watcher, disabling cache 2024-03-04T17:45:30.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.444+0000 7fee3e47d980 20 remove_watcher() i=1 2024-03-04T17:45:30.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.444+0000 7fee3e47d980 20 remove_watcher() i=2 2024-03-04T17:45:30.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.448+0000 7fee3e47d980 20 remove_watcher() i=3 2024-03-04T17:45:30.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.448+0000 7fee3e47d980 20 remove_watcher() i=4 2024-03-04T17:45:30.455 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.452+0000 7fee3e47d980 20 remove_watcher() i=5 2024-03-04T17:45:30.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.452+0000 7fee3e47d980 20 remove_watcher() i=6 2024-03-04T17:45:30.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.456+0000 7fee3e47d980 20 remove_watcher() i=7 2024-03-04T17:45:30.475 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-03-04T17:45:30.507 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:30.507 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:30.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.548+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.548+0000 7f6de55c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.548+0000 7f6de55c5980 20 realm 2024-03-04T17:45:30.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.548+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.548+0000 7f6de55c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.548+0000 7f6de55c5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:30.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.548+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.548+0000 7f6de55c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.548+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.552+0000 7f6de55c5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:30.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.552+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.552+0000 7f6de55c5980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:30.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.552+0000 7f6de55c5980 20 searching for the correct realm 2024-03-04T17:45:30.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.560+0000 7f6de38de640 0 -- 172.21.15.47:0/764973172 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f6dcc06dc20 msgr2=0x7f6dcc054d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:30.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.564+0000 7f6de30dd640 0 -- 172.21.15.47:0/764973172 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f6ddc0cb070 msgr2=0x7f6ddc0f0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:30.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:30.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:30.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:30.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:30.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:30.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:30.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:30.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:30.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 zone default found 2024-03-04T17:45:30.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 4 Realm: () 2024-03-04T17:45:30.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:30.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:30.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:30.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 zonegroup default 2024-03-04T17:45:30.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.572+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.576+0000 7f6de55c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.576+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.576+0000 7f6de55c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.576+0000 7f6de55c5980 20 started sync module instance, tier type = 2024-03-04T17:45:30.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.576+0000 7f6de55c5980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:30.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.596+0000 7f6de55c5980 20 add_watcher() i=0 2024-03-04T17:45:30.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.596+0000 7f6de55c5980 20 add_watcher() i=1 2024-03-04T17:45:30.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.596+0000 7f6de55c5980 20 add_watcher() i=2 2024-03-04T17:45:30.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.596+0000 7f6de55c5980 20 add_watcher() i=3 2024-03-04T17:45:30.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.596+0000 7f6de55c5980 20 add_watcher() i=4 2024-03-04T17:45:30.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.596+0000 7f6de55c5980 20 add_watcher() i=5 2024-03-04T17:45:30.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.596+0000 7f6de55c5980 20 add_watcher() i=6 2024-03-04T17:45:30.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.600+0000 7f6de55c5980 20 add_watcher() i=7 2024-03-04T17:45:30.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.600+0000 7f6de55c5980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:30.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6c99fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:30.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:30.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:30.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 5 note: GC not initialized 2024-03-04T17:45:30.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6c98fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:30.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:30.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 20 Filter name: none 2024-03-04T17:45:30.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6c127fc640 20 reqs_thread_entry: start 2024-03-04T17:45:30.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:30.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:30.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:30.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:30.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 : miss 2024-03-04T17:45:30.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:30.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 info.flags=0x17 2024-03-04T17:45:30.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 to cache LRU end 2024-03-04T17:45:30.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:30.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:30.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.1 2024-03-04T17:45:30.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.604+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.0 2024-03-04T17:45:30.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.608+0000 7f6de55c5980 1 NOTICE: finished shard 0 (0 entries removed) 2024-03-04T17:45:30.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.608+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:30.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.608+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.2 2024-03-04T17:45:30.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.608+0000 7f6de55c5980 1 NOTICE: finished shard 2 (0 entries removed) 2024-03-04T17:45:30.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.608+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.3 2024-03-04T17:45:30.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.608+0000 7f6de55c5980 1 NOTICE: finished shard 3 (0 entries removed) 2024-03-04T17:45:30.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.608+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.4 2024-03-04T17:45:30.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.608+0000 7f6de55c5980 1 NOTICE: finished shard 4 (0 entries removed) 2024-03-04T17:45:30.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.608+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:30.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.612+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.6 2024-03-04T17:45:30.615 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.612+0000 7f6de55c5980 1 NOTICE: finished shard 6 (0 entries removed) 2024-03-04T17:45:30.615 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.612+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.7 2024-03-04T17:45:30.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.612+0000 7f6de55c5980 1 NOTICE: finished shard 7 (0 entries removed) 2024-03-04T17:45:30.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.612+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:30.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.612+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.9 2024-03-04T17:45:30.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.616+0000 7f6de55c5980 1 NOTICE: finished shard 9 (0 entries removed) 2024-03-04T17:45:30.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.616+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:30.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.616+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:30.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.616+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c51f950 obj=check-bucket:e state=0x56453c7cb5c8 s->prefetch_data=0 2024-03-04T17:45:30.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.616+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:30.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.616+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c530bf0 obj=check-bucket:d state=0x56453c7c9b88 s->prefetch_data=0 2024-03-04T17:45:30.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.616+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:30.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.616+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c7d1130 obj=check-bucket:c state=0x56453c7ce918 s->prefetch_data=0 2024-03-04T17:45:30.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.616+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:30.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c7d2890 obj=check-bucket:f state=0x56453c7c7638 s->prefetch_data=0 2024-03-04T17:45:30.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 get_obj_state: setting s->obj_tag to c84iq2vy7wwxndivkdks0uwlll55d1tg 2024-03-04T17:45:30.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 get_obj_state_impl: setting s->olh_tag to 3v6r86bs5wduf4t3um4hizh10z872sbl 2024-03-04T17:45:30.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3v6r86bs5wduf4t3um4hizh10z872sbl 2024-03-04T17:45:30.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 get_obj_state: setting s->obj_tag to cxzjakeacp8uiqwzo2d2nid4jg3cum4j 2024-03-04T17:45:30.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 get_obj_state_impl: setting s->olh_tag to cy2gw9ijypcpwmkg6ot0i5nm58u2t99d 2024-03-04T17:45:30.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=cy2gw9ijypcpwmkg6ot0i5nm58u2t99d 2024-03-04T17:45:30.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 get_obj_state: setting s->obj_tag to 2daxg96utrqaluf921besdxa54q3ayzc 2024-03-04T17:45:30.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 get_obj_state_impl: setting s->olh_tag to a1bfx16i4dhjnmr1x26pon7p251ygah2 2024-03-04T17:45:30.624 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=a1bfx16i4dhjnmr1x26pon7p251ygah2 2024-03-04T17:45:30.624 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 get_obj_state: setting s->obj_tag to 2m1j38aki3oxgie5twlyf248gehxt8bp 2024-03-04T17:45:30.624 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 get_obj_state_impl: setting s->olh_tag to cj8foy7phpa4wyj2bhvoe27ostwniwga 2024-03-04T17:45:30.624 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=cj8foy7phpa4wyj2bhvoe27ostwniwga 2024-03-04T17:45:30.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.620+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:30.626 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:30.628 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:30.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-03-04T17:45:30.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 olh_log_entry: epoch=2 op=3 key=e[z-oU48CEzKLlMKszAoyxmtXkhbeINpL] 2024-03-04T17:45:30.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c51f950 obj=check-bucket:_:z-oU48CEzKLlMKszAoyxmtXkhbeINpL_e state=0x56453c86edc8 s->prefetch_data=0 2024-03-04T17:45:30.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 10 manifest: total_size = 22 2024-03-04T17:45:30.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.2727645327557017881 2024-03-04T17:45:30.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c51f950 obj=check-bucket:_:z-oU48CEzKLlMKszAoyxmtXkhbeINpL_e state=0x56453c86edc8 s->prefetch_data=0 2024-03-04T17:45:30.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 prepare_atomic_modification: state is not atomic. state=0x56453c86edc8 2024-03-04T17:45:30.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:30.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:30.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.624+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:30.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.628+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:30.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.628+0000 7f6de55c5980 20 olh_log_entry: epoch=3 op=3 key=c[93nGojAoCrJnDvfopeUubKepAipfJsc] 2024-03-04T17:45:30.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.628+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c7d1130 obj=check-bucket:_:93nGojAoCrJnDvfopeUubKepAipfJsc_c state=0x56453c870e18 s->prefetch_data=0 2024-03-04T17:45:30.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.628+0000 7f6de55c5980 10 manifest: total_size = 9 2024-03-04T17:45:30.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.628+0000 7f6de55c5980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.8601062623132534092 2024-03-04T17:45:30.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.628+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c7d1130 obj=check-bucket:_:93nGojAoCrJnDvfopeUubKepAipfJsc_c state=0x56453c870e18 s->prefetch_data=0 2024-03-04T17:45:30.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.628+0000 7f6de55c5980 20 prepare_atomic_modification: state is not atomic. state=0x56453c870e18 2024-03-04T17:45:30.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.628+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:30.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:30.635 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 olh_log_entry: epoch=3 op=3 key=d[ckZ1MeIHpWYtf421rxyn1VCdLb8G863] 2024-03-04T17:45:30.635 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c530bf0 obj=check-bucket:_:ckZ1MeIHpWYtf421rxyn1VCdLb8G863_d state=0x56453c8823d8 s->prefetch_data=0 2024-03-04T17:45:30.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:30.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 10 manifest: total_size = 9 2024-03-04T17:45:30.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.922998774271584986 2024-03-04T17:45:30.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c530bf0 obj=check-bucket:_:ckZ1MeIHpWYtf421rxyn1VCdLb8G863_d state=0x56453c8823d8 s->prefetch_data=0 2024-03-04T17:45:30.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 prepare_atomic_modification: state is not atomic. state=0x56453c8823d8 2024-03-04T17:45:30.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:30.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-03-04T17:45:30.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 olh_log_entry: epoch=2 op=3 key=f[81REOGHEuDgGCUDIw.BEqVxIWMzM.yY] 2024-03-04T17:45:30.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.632+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c7d2890 obj=check-bucket:_:81REOGHEuDgGCUDIw.BEqVxIWMzM.yY_f state=0x56453c88c818 s->prefetch_data=0 2024-03-04T17:45:30.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.636+0000 7f6de55c5980 10 manifest: total_size = 11 2024-03-04T17:45:30.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.636+0000 7f6de55c5980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.15337088272239506737 2024-03-04T17:45:30.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.636+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c7d2890 obj=check-bucket:_:81REOGHEuDgGCUDIw.BEqVxIWMzM.yY_f state=0x56453c88c818 s->prefetch_data=0 2024-03-04T17:45:30.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.636+0000 7f6de55c5980 20 prepare_atomic_modification: state is not atomic. state=0x56453c88c818 2024-03-04T17:45:30.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.636+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:30.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.636+0000 7f6cc6ffd640 20 handle_completion(): completion ok for obj=e[z-oU48CEzKLlMKszAoyxmtXkhbeINpL] 2024-03-04T17:45:30.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.636+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c51f950 obj=check-bucket:e state=0x56453c7cb5c8 s->prefetch_data=0 2024-03-04T17:45:30.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.636+0000 7f6de55c5980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3v6r86bs5wduf4t3um4hizh10z872sbl 2024-03-04T17:45:30.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.640+0000 7f6cc6ffd640 20 handle_completion(): completion ok for obj=c[93nGojAoCrJnDvfopeUubKepAipfJsc] 2024-03-04T17:45:30.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.640+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:30.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.640+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:30.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.644+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.8 2024-03-04T17:45:30.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.644+0000 7f6de55c5980 1 NOTICE: finished shard 8 (1 entries removed) 2024-03-04T17:45:30.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.648+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:30.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.648+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:30.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.648+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:30.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.648+0000 7f6cc6ffd640 20 handle_completion(): completion ok for obj=d[ckZ1MeIHpWYtf421rxyn1VCdLb8G863] 2024-03-04T17:45:30.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.648+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.1 2024-03-04T17:45:30.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.648+0000 7f6de55c5980 1 NOTICE: finished shard 1 (1 entries removed) 2024-03-04T17:45:30.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.652+0000 7f6de55c5980 20 get_obj_state: octx=0x56453c7d2890 obj=check-bucket:f state=0x56453c7c7638 s->prefetch_data=0 2024-03-04T17:45:30.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.652+0000 7f6de55c5980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=cj8foy7phpa4wyj2bhvoe27ostwniwga 2024-03-04T17:45:30.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.652+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.5 2024-03-04T17:45:30.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.652+0000 7f6de55c5980 1 NOTICE: finished shard 5 (1 entries removed) 2024-03-04T17:45:30.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.652+0000 7f6cc67fc640 20 handle_completion(): completion ok for obj=f[81REOGHEuDgGCUDIw.BEqVxIWMzM.yY] 2024-03-04T17:45:30.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.656+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:30.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.660+0000 7f6de55c5980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.1.10 2024-03-04T17:45:30.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.660+0000 7f6de55c5980 1 NOTICE: finished shard 10 (1 entries removed) 2024-03-04T17:45:30.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.660+0000 7f6de55c5980 1 NOTICE: finished all shards (4 entries removed) 2024-03-04T17:45:30.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.660+0000 7f6c99fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:30.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.664+0000 7f6de55c5980 20 remove_watcher() i=0 2024-03-04T17:45:30.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.664+0000 7f6de55c5980 2 removed watcher, disabling cache 2024-03-04T17:45:30.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.668+0000 7f6de55c5980 20 remove_watcher() i=1 2024-03-04T17:45:30.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.668+0000 7f6de55c5980 20 remove_watcher() i=2 2024-03-04T17:45:30.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.672+0000 7f6de55c5980 20 remove_watcher() i=3 2024-03-04T17:45:30.676 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.672+0000 7f6de55c5980 20 remove_watcher() i=4 2024-03-04T17:45:30.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.676+0000 7f6de55c5980 20 remove_watcher() i=5 2024-03-04T17:45:30.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.676+0000 7f6de55c5980 20 remove_watcher() i=6 2024-03-04T17:45:30.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:30.680+0000 7f6de55c5980 20 remove_watcher() i=7 2024-03-04T17:45:30.700 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:30.700 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:30.744 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:30.744 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:31.130 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-03-04T17:45:31.162 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:31.162 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:31.206 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.204+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.204+0000 7f22e947c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.204+0000 7f22e947c980 20 realm 2024-03-04T17:45:31.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.204+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.204+0000 7f22e947c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.204+0000 7f22e947c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:31.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.204+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.208+0000 7f22e947c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.208+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.208+0000 7f22e947c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:31.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.208+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.208+0000 7f22e947c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:31.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.208+0000 7f22e947c980 20 searching for the correct realm 2024-03-04T17:45:31.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.216+0000 7f22e6793640 0 -- 172.21.15.47:0/1506818683 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f22dc0e1b60 msgr2=0x7f22dc0c6950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:31.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.428+0000 7f22e6793640 0 -- 172.21.15.47:0/1506818683 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55d06c46b520 msgr2=0x55d06c46d930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:31.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:45:31.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:45:31.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:31.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:31.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:31.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:31.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:31.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:31.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 zone default found 2024-03-04T17:45:31.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 4 Realm: () 2024-03-04T17:45:31.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:45:31.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:45:31.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:31.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 zonegroup default 2024-03-04T17:45:31.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.444+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.448+0000 7f22e947c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.448+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.448+0000 7f22e947c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.448+0000 7f22e947c980 20 started sync module instance, tier type = 2024-03-04T17:45:31.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.448+0000 7f22e947c980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:45:31.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.472+0000 7f22e947c980 20 add_watcher() i=0 2024-03-04T17:45:31.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.472+0000 7f22e947c980 20 add_watcher() i=1 2024-03-04T17:45:31.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.472+0000 7f22e947c980 20 add_watcher() i=2 2024-03-04T17:45:31.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.472+0000 7f22e947c980 20 add_watcher() i=3 2024-03-04T17:45:31.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.472+0000 7f22e947c980 20 add_watcher() i=4 2024-03-04T17:45:31.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.472+0000 7f22e947c980 20 add_watcher() i=5 2024-03-04T17:45:31.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.472+0000 7f22e947c980 20 add_watcher() i=6 2024-03-04T17:45:31.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.472+0000 7f22e947c980 20 add_watcher() i=7 2024-03-04T17:45:31.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.472+0000 7f22e947c980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:31.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.476+0000 7f219d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:31.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:31.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:31.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 5 note: GC not initialized 2024-03-04T17:45:31.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f2117fff640 20 reqs_thread_entry: start 2024-03-04T17:45:31.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 init_complete bucket index max shards: 11 2024-03-04T17:45:31.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 Filter name: none 2024-03-04T17:45:31.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f2115ffb640 20 reqs_thread_entry: start 2024-03-04T17:45:31.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:31.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:31.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:31.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.480+0000 7f22e947c980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:31.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.484+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.484+0000 7f22e947c980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:31.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.484+0000 7f22e947c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:31.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.484+0000 7f22e947c980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.484+0000 7f22e947c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.488+0000 7f219d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:31.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.492+0000 7f22e947c980 20 remove_watcher() i=0 2024-03-04T17:45:31.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.492+0000 7f22e947c980 2 removed watcher, disabling cache 2024-03-04T17:45:31.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.492+0000 7f22e947c980 20 remove_watcher() i=1 2024-03-04T17:45:31.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.496+0000 7f22e947c980 20 remove_watcher() i=2 2024-03-04T17:45:31.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.500+0000 7f22e947c980 20 remove_watcher() i=3 2024-03-04T17:45:31.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.500+0000 7f22e947c980 20 remove_watcher() i=4 2024-03-04T17:45:31.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.504+0000 7f22e947c980 20 remove_watcher() i=5 2024-03-04T17:45:31.509 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.504+0000 7f22e947c980 20 remove_watcher() i=6 2024-03-04T17:45:31.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:45:31.508+0000 7f22e947c980 20 remove_watcher() i=7 2024-03-04T17:45:31.529 INFO:tasks.workunit.client.0.smithi047.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-03-04T17:45:31.529 INFO:tasks.workunit.client.0.smithi047.stderr:Deleting bucket check-bucket 2024-03-04T17:45:31.530 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:31.531 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:31.580 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:31.580 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:45:32.098 INFO:tasks.workunit.client.0.smithi047.stderr:Completed bucket check tests 2024-03-04T17:45:32.099 INFO:tasks.workunit.client.0.smithi047.stdout:Could not connect to the endpoint URL: "http://localhost:80/" 2024-03-04T17:45:32.099 INFO:tasks.workunit.client.0.smithi047.stdout:Could not connect to the endpoint URL: "http://localhost:8000/" 2024-03-04T17:45:32.099 INFO:tasks.workunit.client.0.smithi047.stdout:connected to https://localhost:443 2024-03-04T17:45:32.201 INFO:tasks.workunit.client.0.smithi047.stderr:+ deactivate 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ export PATH 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ unset _OLD_VIRTUAL_PATH 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' -n '' ']' 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ hash -r 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' -n '' ']' 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ unset VIRTUAL_ENV 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' '!' '' = nondestructive ']' 2024-03-04T17:45:32.202 INFO:tasks.workunit.client.0.smithi047.stderr:+ unset -f deactivate 2024-03-04T17:45:32.203 INFO:tasks.workunit.client.0.smithi047.stdout:OK. 2024-03-04T17:45:32.204 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo OK. 2024-03-04T17:45:32.204 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:45:32.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:45:32.219 INFO:tasks.workunit:Stopping ['rgw/run-bucket-check.sh'] on client.0... 2024-03-04T17:45:32.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T17:45:32.641 DEBUG:teuthology.parallel:result is None 2024-03-04T17:45:32.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:45:32.654 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:45:32.654 DEBUG:teuthology.orchestra.run.smithi047:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:45:32.716 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:45:32.716 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T17:45:32.727 INFO:tasks.workunit:Pulling workunits from ref 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:45:32.727 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T17:45:32.727 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:45:32.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:45:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T17:45:32.762 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:45:32.814 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T17:45:32.814 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T17:45:32.862 INFO:tasks.workunit:timeout=3h 2024-03-04T17:45:32.862 INFO:tasks.workunit:cleanup=True 2024-03-04T17:45:32.863 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/pritha-srivastava/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:45:32.912 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T17:46:47.708 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to '3aec390336fdfdc8f927818b718cfb0361bd47c8'. 2024-03-04T17:46:47.708 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:46:47.708 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:46:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T17:46:47.710 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:46:47.710 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at 3aec390336f qa/d4n: Start redis up before rgw 2024-03-04T17:46:47.714 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T17:46:47.763 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T17:46:47.766 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:46:47.766 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T17:46:47.829 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T17:46:47.894 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T17:46:47.941 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:46:47.943 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:46:47.943 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T17:46:47.985 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:46:47.990 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:46:47.990 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T17:46:48.038 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2024-03-04T17:46:48.039 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2024-03-04T17:46:48.039 DEBUG:teuthology.orchestra.run.smithi047:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2024-03-04T17:46:48.103 INFO:tasks.workunit.client.0.smithi047.stdout:Running main() from gmock_main.cc 2024-03-04T17:46:48.103 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 11 tests from 1 test suite. 2024-03-04T17:46:48.104 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:46:48.104 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 11 tests from ClsLock 2024-03-04T17:46:48.104 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestMultiLocking 2024-03-04T17:46:49.878 INFO:tasks.workunit.client.0.smithi047.stdout:lock: mylock 2024-03-04T17:46:49.879 INFO:tasks.workunit.client.0.smithi047.stdout: lock_type: exclusive 2024-03-04T17:46:49.879 INFO:tasks.workunit.client.0.smithi047.stdout: tag: 2024-03-04T17:46:49.879 INFO:tasks.workunit.client.0.smithi047.stdout: lockers: 2024-03-04T17:46:49.879 INFO:tasks.workunit.client.0.smithi047.stdout: client.4952 expiration=2024-03-04T17:48:49.874805+0000 addr=v1:172.21.15.47:0/3054600580 cookie= 2024-03-04T17:46:49.885 INFO:tasks.workunit.client.0.smithi047.stdout:lock: mylock 2024-03-04T17:46:49.885 INFO:tasks.workunit.client.0.smithi047.stdout: lock_type: shared 2024-03-04T17:46:49.885 INFO:tasks.workunit.client.0.smithi047.stdout: tag: 2024-03-04T17:46:49.885 INFO:tasks.workunit.client.0.smithi047.stdout: lockers: 2024-03-04T17:46:49.885 INFO:tasks.workunit.client.0.smithi047.stdout: client.4952 expiration=2024-03-04T17:48:49.885898+0000 addr=v1:172.21.15.47:0/3054600580 cookie= 2024-03-04T17:46:49.885 INFO:tasks.workunit.client.0.smithi047.stdout: client.4955 expiration=0.000000 addr=v1:172.21.15.47:0/579230144 cookie= 2024-03-04T17:46:49.887 INFO:tasks.workunit.client.0.smithi047.stdout:lock: mylock 2024-03-04T17:46:49.887 INFO:tasks.workunit.client.0.smithi047.stdout: lock_type: shared 2024-03-04T17:46:49.887 INFO:tasks.workunit.client.0.smithi047.stdout: tag: 2024-03-04T17:46:49.888 INFO:tasks.workunit.client.0.smithi047.stdout: lockers: 2024-03-04T17:46:49.888 INFO:tasks.workunit.client.0.smithi047.stdout: client.4955 expiration=0.000000 addr=v1:172.21.15.47:0/579230144 cookie= 2024-03-04T17:46:50.831 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestMultiLocking (2728 ms) 2024-03-04T17:46:50.831 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestMeta 2024-03-04T17:46:52.918 INFO:tasks.workunit.client.0.smithi047.stdout:lock: mylock 2024-03-04T17:46:52.918 INFO:tasks.workunit.client.0.smithi047.stdout: lock_type: shared 2024-03-04T17:46:52.918 INFO:tasks.workunit.client.0.smithi047.stdout: tag: 2024-03-04T17:46:52.918 INFO:tasks.workunit.client.0.smithi047.stdout: lockers: 2024-03-04T17:46:52.918 INFO:tasks.workunit.client.0.smithi047.stdout: client.4967 expiration=0.000000 addr=v1:172.21.15.47:0/3199882041 cookie= 2024-03-04T17:46:52.921 INFO:tasks.workunit.client.0.smithi047.stdout:lock: mylock 2024-03-04T17:46:52.921 INFO:tasks.workunit.client.0.smithi047.stdout: lock_type: exclusive 2024-03-04T17:46:52.921 INFO:tasks.workunit.client.0.smithi047.stdout: tag: new_tag 2024-03-04T17:46:52.921 INFO:tasks.workunit.client.0.smithi047.stdout: lockers: 2024-03-04T17:46:52.921 INFO:tasks.workunit.client.0.smithi047.stdout: client.5166 expiration=0.000000 addr=v1:172.21.15.47:0/3412403562 cookie= 2024-03-04T17:46:53.871 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestMeta (3039 ms) 2024-03-04T17:46:53.871 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestCookie 2024-03-04T17:46:55.933 INFO:tasks.workunit.client.0.smithi047.stdout:lock: mylock 2024-03-04T17:46:55.933 INFO:tasks.workunit.client.0.smithi047.stdout: lock_type: exclusive 2024-03-04T17:46:55.933 INFO:tasks.workunit.client.0.smithi047.stdout: tag: 2024-03-04T17:46:55.933 INFO:tasks.workunit.client.0.smithi047.stdout: lockers: 2024-03-04T17:46:55.938 INFO:tasks.workunit.client.0.smithi047.stdout:lock: mylock 2024-03-04T17:46:55.938 INFO:tasks.workunit.client.0.smithi047.stdout: lock_type: shared 2024-03-04T17:46:55.938 INFO:tasks.workunit.client.0.smithi047.stdout: tag: 2024-03-04T17:46:55.939 INFO:tasks.workunit.client.0.smithi047.stdout: lockers: 2024-03-04T17:46:55.939 INFO:tasks.workunit.client.0.smithi047.stdout: client.5175 expiration=0.000000 addr=v1:172.21.15.47:0/1566533324 cookie= 2024-03-04T17:46:55.939 INFO:tasks.workunit.client.0.smithi047.stdout: client.5175 expiration=0.000000 addr=v1:172.21.15.47:0/1566533324 cookie=new cookie 2024-03-04T17:46:56.894 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestCookie (3024 ms) 2024-03-04T17:46:56.894 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestMultipleLocks 2024-03-04T17:47:00.139 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestMultipleLocks (3244 ms) 2024-03-04T17:47:00.139 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestLockDuration 2024-03-04T17:47:08.180 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestLockDuration (8042 ms) 2024-03-04T17:47:08.180 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestAssertLocked 2024-03-04T17:47:11.208 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestAssertLocked (3024 ms) 2024-03-04T17:47:11.208 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestSetCookie 2024-03-04T17:47:14.258 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestSetCookie (3054 ms) 2024-03-04T17:47:14.259 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestRenew 2024-03-04T17:47:34.496 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestRenew (20234 ms) 2024-03-04T17:47:34.497 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2024-03-04T17:47:41.536 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7043 ms) 2024-03-04T17:47:41.536 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2024-03-04T17:47:49.593 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8058 ms) 2024-03-04T17:47:49.593 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2024-03-04T17:47:57.655 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8062 ms) 2024-03-04T17:47:57.655 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 11 tests from ClsLock (69552 ms total) 2024-03-04T17:47:57.655 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:47:57.655 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:47:57.655 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 11 tests from 1 test suite ran. (69552 ms total) 2024-03-04T17:47:57.655 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 11 tests. 2024-03-04T17:47:57.662 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:47:57.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:47:57.717 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2024-03-04T17:47:57.718 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2024-03-04T17:47:57.718 DEBUG:teuthology.orchestra.run.smithi047:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2024-03-04T17:47:57.787 INFO:tasks.workunit.client.0.smithi047.stdout:Running main() from gmock_main.cc 2024-03-04T17:47:57.787 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-04T17:47:57.787 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:47:57.787 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 4 tests from cls_log 2024-03-04T17:47:57.787 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_log.test_log_add_same_time 2024-03-04T17:48:00.684 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_log.test_log_add_same_time (2898 ms) 2024-03-04T17:48:00.685 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_log.test_log_add_different_time 2024-03-04T17:48:03.721 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_log.test_log_add_different_time (3036 ms) 2024-03-04T17:48:03.721 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_log.trim_by_time 2024-03-04T17:48:06.780 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_log.trim_by_time (3059 ms) 2024-03-04T17:48:06.780 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_log.trim_by_marker 2024-03-04T17:48:09.827 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_log.trim_by_marker (3047 ms) 2024-03-04T17:48:09.827 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 4 tests from cls_log (12040 ms total) 2024-03-04T17:48:09.827 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:48:09.827 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:48:09.827 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 4 tests from 1 test suite ran. (12040 ms total) 2024-03-04T17:48:09.827 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 4 tests. 2024-03-04T17:48:09.833 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:48:09.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:48:09.892 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2024-03-04T17:48:09.895 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2024-03-04T17:48:09.895 DEBUG:teuthology.orchestra.run.smithi047:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2024-03-04T17:48:09.964 INFO:tasks.workunit.client.0.smithi047.stdout:Running main() from gmock_main.cc 2024-03-04T17:48:09.965 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 10 tests from 1 test suite. 2024-03-04T17:48:09.965 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:48:09.965 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 10 tests from cls_refcount 2024-03-04T17:48:09.965 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.test_implicit 2024-03-04T17:48:12.861 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.test_implicit (2898 ms) 2024-03-04T17:48:12.861 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.test_implicit_idempotent 2024-03-04T17:48:15.890 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.test_implicit_idempotent (3028 ms) 2024-03-04T17:48:15.890 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.test_put_snap 2024-03-04T17:48:24.974 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.test_put_snap (9084 ms) 2024-03-04T17:48:24.974 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.test_explicit 2024-03-04T17:48:28.017 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.test_explicit (3043 ms) 2024-03-04T17:48:28.017 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.set 2024-03-04T17:48:31.046 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.set (3029 ms) 2024-03-04T17:48:31.046 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.test_implicit_ec 2024-03-04T17:48:38.071 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.test_implicit_ec (7025 ms) 2024-03-04T17:48:38.071 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.test_implicit_idempotent_ec 2024-03-04T17:48:45.172 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.test_implicit_idempotent_ec (7101 ms) 2024-03-04T17:48:45.173 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.test_put_snap_ec 2024-03-04T17:48:57.300 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.test_put_snap_ec (12128 ms) 2024-03-04T17:48:57.301 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.test_explicit_ec 2024-03-04T17:49:04.370 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.test_explicit_ec (7070 ms) 2024-03-04T17:49:04.370 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_refcount.set_ec 2024-03-04T17:49:11.436 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_refcount.set_ec (7066 ms) 2024-03-04T17:49:11.436 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 10 tests from cls_refcount (61473 ms total) 2024-03-04T17:49:11.436 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:49:11.437 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:49:11.437 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 10 tests from 1 test suite ran. (61473 ms total) 2024-03-04T17:49:11.437 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 10 tests. 2024-03-04T17:49:11.444 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:49:11.445 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:49:11.505 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2024-03-04T17:49:11.506 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2024-03-04T17:49:11.506 DEBUG:teuthology.orchestra.run.smithi047:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2024-03-04T17:49:11.570 INFO:tasks.workunit.client.0.smithi047.stdout:Running main() from gmock_main.cc 2024-03-04T17:49:11.570 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 16 tests from 1 test suite. 2024-03-04T17:49:11.570 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:49:11.570 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 16 tests from cls_rgw 2024-03-04T17:49:13.494 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.index_basic 2024-03-04T17:49:13.582 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.index_basic (88 ms) 2024-03-04T17:49:13.582 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2024-03-04T17:49:13.661 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (79 ms) 2024-03-04T17:49:13.661 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.index_remove_object 2024-03-04T17:49:13.755 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.index_remove_object (94 ms) 2024-03-04T17:49:13.755 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.index_suggest 2024-03-04T17:49:16.183 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.index_suggest (2428 ms) 2024-03-04T17:49:16.183 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.index_suggest_complete 2024-03-04T17:49:16.199 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.index_suggest_complete (16 ms) 2024-03-04T17:49:16.199 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.index_list 2024-03-04T17:49:16.275 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.index_list (77 ms) 2024-03-04T17:49:16.275 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.index_list_delimited 2024-03-04T17:50:15.506 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.index_list_delimited (59230 ms) 2024-03-04T17:50:15.506 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.bi_list 2024-03-04T17:50:15.717 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.bi_list (212 ms) 2024-03-04T17:50:15.717 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.gc_set 2024-03-04T17:50:15.747 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.gc_set (31 ms) 2024-03-04T17:50:15.748 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.gc_list 2024-03-04T17:50:15.984 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.gc_list (237 ms) 2024-03-04T17:50:15.985 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.gc_defer 2024-03-04T17:50:23.130 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.gc_defer (7145 ms) 2024-03-04T17:50:23.130 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.usage_basic 2024-03-04T17:50:23.335 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.usage_basic (206 ms) 2024-03-04T17:50:23.335 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2024-03-04T17:50:23.338 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.usage_clear_no_obj (3 ms) 2024-03-04T17:50:23.338 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.usage_clear 2024-03-04T17:50:23.547 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.usage_clear (208 ms) 2024-03-04T17:50:23.547 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.bi_log_trim 2024-03-04T17:50:23.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.bi_log_trim (131 ms) 2024-03-04T17:50:23.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw.index_racing_removes 2024-03-04T17:50:23.713 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw.index_racing_removes (36 ms) 2024-03-04T17:50:24.153 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 16 tests from cls_rgw (70221 ms total) 2024-03-04T17:50:24.153 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:50:24.153 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:50:24.153 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 16 tests from 1 test suite ran. (72584 ms total) 2024-03-04T17:50:24.153 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 16 tests. 2024-03-04T17:50:24.161 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:50:24.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:50:24.215 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_gc.sh on client.0... 2024-03-04T17:50:24.216 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2024-03-04T17:50:24.216 DEBUG:teuthology.orchestra.run.smithi047:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2024-03-04T17:50:24.282 INFO:tasks.workunit.client.0.smithi047.stdout:Running main() from gmock_main.cc 2024-03-04T17:50:24.282 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 9 tests from 1 test suite. 2024-03-04T17:50:24.282 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:50:24.282 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 9 tests from cls_rgw_gc 2024-03-04T17:50:24.282 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_gc.init 2024-03-04T17:50:26.161 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_gc.init (1880 ms) 2024-03-04T17:50:26.162 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2024-03-04T17:50:26.205 INFO:tasks.workunit.client.0.smithi047.stderr:[ ] list info tag = chain-0 2024-03-04T17:50:26.206 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (44 ms) 2024-03-04T17:50:26.206 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2024-03-04T17:50:26.218 INFO:tasks.workunit.client.0.smithi047.stderr:[ ] next_marker is: = 0/4263 2024-03-04T17:50:26.219 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (14 ms) 2024-03-04T17:50:26.219 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2024-03-04T17:50:26.236 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (17 ms) 2024-03-04T17:50:26.236 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2024-03-04T17:50:26.251 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (15 ms) 2024-03-04T17:50:26.252 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2024-03-04T17:50:26.265 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (14 ms) 2024-03-04T17:50:26.266 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2024-03-04T17:50:26.283 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (17 ms) 2024-03-04T17:50:26.283 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2024-03-04T17:50:26.299 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (16 ms) 2024-03-04T17:50:26.299 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_gc.finalize 2024-03-04T17:50:27.180 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_gc.finalize (878 ms) 2024-03-04T17:50:27.180 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 9 tests from cls_rgw_gc (2895 ms total) 2024-03-04T17:50:27.180 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:50:27.180 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:50:27.180 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 9 tests from 1 test suite ran. (2895 ms total) 2024-03-04T17:50:27.180 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 9 tests. 2024-03-04T17:50:27.183 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:50:27.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:50:27.239 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_stats.sh on client.0... 2024-03-04T17:50:27.240 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2024-03-04T17:50:27.241 DEBUG:teuthology.orchestra.run.smithi047:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2024-03-04T17:50:27.316 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:50:27.317 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:50:27.317 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 1 test from 1 test suite. 2024-03-04T17:50:27.317 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:50:29.208 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 1 test from cls_rgw_stats 2024-03-04T17:50:29.208 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] cls_rgw_stats.simulate 2024-03-04T17:50:29.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.261+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=QL5Z2MC3AQZF type=del size=833680 2024-03-04T17:50:29.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.261+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=QPS515F7PG0V type=del size=3764353 2024-03-04T17:50:29.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.269+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~MF4BRKMPQJ95.0 size=824534 2024-03-04T17:50:29.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.269+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=MF4BRKMPQJ95 tag=QQ3ID6WGA5K8 type=write size=1873110 2024-03-04T17:50:29.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.269+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=E3AIGTCW6BH3 type=del size=2664216 2024-03-04T17:50:29.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.277+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~VJ2BRH9WJBP3.0 size=764579 2024-03-04T17:50:29.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.277+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=VJ2BRH9WJBP3 tag=G3C03BCGHH1M type=write size=1813155 2024-03-04T17:50:29.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.281+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=YRH467LYRP5C type=del size=2172176 2024-03-04T17:50:29.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.285+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~YPWEYWS5WYR2.0 size=1048576 2024-03-04T17:50:29.287 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.289+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~YPWEYWS5WYR2.1 size=1048576 2024-03-04T17:50:29.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.289+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~YPWEYWS5WYR2.2 size=221573 2024-03-04T17:50:29.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.289+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=YPWEYWS5WYR2 tag=JQPEFRY13DBU type=write size=3367301 2024-03-04T17:50:29.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.293+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=J69MX5AVAN8Y type=del size=4080847 2024-03-04T17:50:29.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.297+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=JI33XMIGXVPC type=del size=2621021 2024-03-04T17:50:29.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.301+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=6T1ZPEBHH2QJ type=del size=4004224 2024-03-04T17:50:29.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.305+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=W85SJ022BAS6 type=del size=715780 2024-03-04T17:50:29.308 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.309+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=0ZX0QAPIZBTZ type=del size=3138243 2024-03-04T17:50:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.317+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~5UZX41V3NDV7.0 size=426951 2024-03-04T17:50:29.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.317+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=5UZX41V3NDV7 tag=0XGKA9X5SX9B type=write size=1475527 2024-03-04T17:50:29.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.321+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=JIRDXMKHIG49 type=del size=3667163 2024-03-04T17:50:29.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.325+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=G2AA9MDD8NPW type=del size=188417 2024-03-04T17:50:29.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.333+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~I3NNQKJ1LQ84.0 size=739023 2024-03-04T17:50:29.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.333+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=I3NNQKJ1LQ84 tag=GTIWBTN8IZ2N type=write size=1787599 2024-03-04T17:50:29.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.333+0000 7fda08af98c0 1 < canceled operation key=HCA7QLGM2O4O tag=QL5Z2MC3AQZF type=del size=833680 2024-03-04T17:50:29.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.341+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~HJTG0QCQKBB5.0 size=1048576 2024-03-04T17:50:29.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.345+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~HJTG0QCQKBB5.1 size=1048576 2024-03-04T17:50:29.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.349+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~HJTG0QCQKBB5.2 size=413564 2024-03-04T17:50:29.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.349+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=HJTG0QCQKBB5 tag=DDZMCPVAQQA3 type=write size=3559292 2024-03-04T17:50:29.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.353+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=QPS515F7PG0V type=del 2024-03-04T17:50:29.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.357+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=J535QG5M8NYJ type=del size=2604565 2024-03-04T17:50:29.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.361+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~MF4BRKMPQJ95.0 size=824534 2024-03-04T17:50:29.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.365+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=MF4BRKMPQJ95 tag=QQ3ID6WGA5K8 type=write size=1873110 2024-03-04T17:50:29.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.369+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=3HD88UWK073M type=del size=3031745 2024-03-04T17:50:29.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.373+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=E3AIGTCW6BH3 type=del 2024-03-04T17:50:29.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.381+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=H7F3O3RNYSFO type=del size=1550098 2024-03-04T17:50:29.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.385+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~VJ2BRH9WJBP3.0 size=764579 2024-03-04T17:50:29.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.393+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=VJ2BRH9WJBP3 tag=G3C03BCGHH1M type=write size=1813155 2024-03-04T17:50:29.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.401+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~46BPK0U7OURT.0 size=1048576 2024-03-04T17:50:29.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.401+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~46BPK0U7OURT.1 size=211367 2024-03-04T17:50:29.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.401+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=46BPK0U7OURT tag=I76AWMWYOKLY type=write size=2308519 2024-03-04T17:50:29.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.409+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=YRH467LYRP5C type=del 2024-03-04T17:50:29.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.413+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~9QVAHAJXO0X6.0 size=1048576 2024-03-04T17:50:29.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.417+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~9QVAHAJXO0X6.1 size=1048576 2024-03-04T17:50:29.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.421+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~9QVAHAJXO0X6.2 size=1021738 2024-03-04T17:50:29.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.421+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=9QVAHAJXO0X6 tag=V2CG1BIVPHAT type=write size=4167466 2024-03-04T17:50:29.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.425+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~YPWEYWS5WYR2.0 size=1048576 2024-03-04T17:50:29.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.429+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~YPWEYWS5WYR2.1 size=1048576 2024-03-04T17:50:29.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.429+0000 7fda08af98c0 1 < canceled part key=_multipart_DJCCQ6S0E7UG.2~YPWEYWS5WYR2.2 size=221573 2024-03-04T17:50:29.432 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.433+0000 7fda08af98c0 1 < canceled multipart upload key=DJCCQ6S0E7UG upload=YPWEYWS5WYR2 tag=JQPEFRY13DBU type=write size=3367301 2024-03-04T17:50:29.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.437+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=V3UUF1ACQCHR type=del size=4029479 2024-03-04T17:50:29.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.441+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=J69MX5AVAN8Y type=del 2024-03-04T17:50:29.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.445+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=2Y2BCWX2CWCA type=del size=2384028 2024-03-04T17:50:29.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.449+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=JI33XMIGXVPC type=del 2024-03-04T17:50:29.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.453+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=16LJHARBSB9K type=del size=2091985 2024-03-04T17:50:29.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.457+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=6T1ZPEBHH2QJ type=del 2024-03-04T17:50:29.458 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.461+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=3RUUP8OB8HJY type=del size=708176 2024-03-04T17:50:29.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.465+0000 7fda08af98c0 1 < suggested update operation key=L0FVKWD7EQRW tag=W85SJ022BAS6 type=del size=715780 2024-03-04T17:50:29.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.473+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~7YIRZU8SIKFU.0 size=637653 2024-03-04T17:50:29.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.473+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=7YIRZU8SIKFU tag=I43MI3ZSCZ73 type=write size=1686229 2024-03-04T17:50:29.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.477+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=0ZX0QAPIZBTZ type=del 2024-03-04T17:50:29.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.485+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~4WIO8Z3FZUJV.0 size=1048576 2024-03-04T17:50:29.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.485+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~4WIO8Z3FZUJV.1 size=1048576 2024-03-04T17:50:29.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.489+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~4WIO8Z3FZUJV.2 size=926897 2024-03-04T17:50:29.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.489+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=4WIO8Z3FZUJV tag=M7ZWR1TY8XUC type=write size=4072625 2024-03-04T17:50:29.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.493+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~5UZX41V3NDV7.0 size=426951 2024-03-04T17:50:29.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.497+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=5UZX41V3NDV7 tag=0XGKA9X5SX9B type=write size=1475527 2024-03-04T17:50:29.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.501+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=ZXY1TEKZDLSQ type=del size=1666245 2024-03-04T17:50:29.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.505+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=JIRDXMKHIG49 type=del 2024-03-04T17:50:29.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.509+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=93GQMJKIRJBW type=del size=418013 2024-03-04T17:50:29.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.513+0000 7fda08af98c0 -1 < failed to complete operation key=L0FVKWD7EQRW tag=G2AA9MDD8NPW type=del size=188417: (22) Invalid argument 2024-03-04T17:50:29.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.517+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=DL52E1QSJOVD type=del size=1797155 2024-03-04T17:50:29.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.521+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~I3NNQKJ1LQ84.0 size=739023 2024-03-04T17:50:29.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.525+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=I3NNQKJ1LQ84 tag=GTIWBTN8IZ2N type=write size=1787599 2024-03-04T17:50:29.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.533+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~AGGBGYGE7ZZ7.0 size=1048576 2024-03-04T17:50:29.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.537+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~AGGBGYGE7ZZ7.1 size=805221 2024-03-04T17:50:29.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.537+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=AGGBGYGE7ZZ7 tag=P9VBWM2THRR9 type=write size=2902373 2024-03-04T17:50:29.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.541+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~HJTG0QCQKBB5.0 size=1048576 2024-03-04T17:50:29.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.541+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~HJTG0QCQKBB5.1 size=1048576 2024-03-04T17:50:29.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.545+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~HJTG0QCQKBB5.2 size=413564 2024-03-04T17:50:29.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.553+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=HJTG0QCQKBB5 tag=DDZMCPVAQQA3 type=write size=3559292 2024-03-04T17:50:29.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.557+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=HFJMX04EW7DY type=del size=1163489 2024-03-04T17:50:29.563 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.565+0000 7fda08af98c0 1 < suggested update operation key=V1LDB2HXZ7EF tag=J535QG5M8NYJ type=del size=2604565 2024-03-04T17:50:29.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.573+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~2RF8XIRP25Z9.0 size=1048576 2024-03-04T17:50:29.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.573+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~2RF8XIRP25Z9.1 size=1048576 2024-03-04T17:50:29.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.577+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~2RF8XIRP25Z9.2 size=931537 2024-03-04T17:50:29.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.577+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=2RF8XIRP25Z9 tag=LP84OAIFEQA6 type=write size=4077265 2024-03-04T17:50:29.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.585+0000 7fda08af98c0 1 < suggested update operation key=XG5I3SDRE5JM tag=3HD88UWK073M type=del size=3031745 2024-03-04T17:50:29.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.593+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~HJK9RYPYKG4J.0 size=1048576 2024-03-04T17:50:29.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.593+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~HJK9RYPYKG4J.1 size=1048576 2024-03-04T17:50:29.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.597+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~HJK9RYPYKG4J.2 size=471294 2024-03-04T17:50:29.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.597+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=HJK9RYPYKG4J tag=EQQH2VK1BRFA type=write size=3617022 2024-03-04T17:50:29.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.601+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=H7F3O3RNYSFO type=del 2024-03-04T17:50:29.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.605+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=Q8OU4FQ7MLZ0 type=write size=883302 2024-03-04T17:50:29.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.609+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~46BPK0U7OURT.0 size=1048576 2024-03-04T17:50:29.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.609+0000 7fda08af98c0 1 < canceled part key=_multipart_3UWSL7NJLACY.2~46BPK0U7OURT.1 size=211367 2024-03-04T17:50:29.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.617+0000 7fda08af98c0 1 < canceled multipart upload key=3UWSL7NJLACY upload=46BPK0U7OURT tag=I76AWMWYOKLY type=write size=2308519 2024-03-04T17:50:29.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.621+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=FEYPEUM25H2R type=del size=2967233 2024-03-04T17:50:29.624 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.625+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~9QVAHAJXO0X6.0 size=1048576 2024-03-04T17:50:29.624 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.625+0000 7fda08af98c0 1 < canceled part key=_multipart_FMPZ2548LBBF.2~9QVAHAJXO0X6.1 size=1048576 2024-03-04T17:50:29.627 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.629+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~9QVAHAJXO0X6.2 size=1021738 2024-03-04T17:50:29.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.633+0000 7fda08af98c0 1 < canceled multipart upload key=FMPZ2548LBBF upload=9QVAHAJXO0X6 tag=V2CG1BIVPHAT type=write size=4167466 2024-03-04T17:50:29.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.645+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~KINXD4WTM9EO.0 size=1048576 2024-03-04T17:50:29.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.649+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~KINXD4WTM9EO.1 size=970265 2024-03-04T17:50:29.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.649+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=KINXD4WTM9EO tag=JSK50KQ3HL7P type=write size=3067417 2024-03-04T17:50:29.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.653+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=V3UUF1ACQCHR type=del 2024-03-04T17:50:29.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.657+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~63VH0NOABB4T.0 size=1048576 2024-03-04T17:50:29.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.661+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~63VH0NOABB4T.1 size=987450 2024-03-04T17:50:29.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.661+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=63VH0NOABB4T tag=69W5OFY8OUHY type=write size=3084602 2024-03-04T17:50:29.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.669+0000 7fda08af98c0 1 < suggested update operation key=BYQFZSJ3WH07 tag=2Y2BCWX2CWCA type=del size=2384028 2024-03-04T17:50:29.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.677+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~D34W27JM14KB.0 size=1048576 2024-03-04T17:50:29.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.681+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~D34W27JM14KB.1 size=224109 2024-03-04T17:50:29.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.681+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=D34W27JM14KB tag=O95V0TBEY9BG type=write size=2321261 2024-03-04T17:50:29.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.685+0000 7fda08af98c0 -1 < failed to complete operation key=BYQFZSJ3WH07 tag=16LJHARBSB9K type=del size=2091985: (22) Invalid argument 2024-03-04T17:50:29.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.689+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=GU10N656201T type=del size=1550565 2024-03-04T17:50:29.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.697+0000 7fda08af98c0 1 < suggested update operation key=1OZKPHMPV3OQ tag=3RUUP8OB8HJY type=del size=708176 2024-03-04T17:50:29.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.701+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=9554JFH3C2G6 type=del size=1722906 2024-03-04T17:50:29.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.705+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~7YIRZU8SIKFU.0 size=637653 2024-03-04T17:50:29.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.709+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=7YIRZU8SIKFU tag=I43MI3ZSCZ73 type=write size=1686229 2024-03-04T17:50:29.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.713+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=S3NSWE5S3K3X type=del size=3240472 2024-03-04T17:50:29.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.717+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~4WIO8Z3FZUJV.0 size=1048576 2024-03-04T17:50:29.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.721+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~4WIO8Z3FZUJV.1 size=1048576 2024-03-04T17:50:29.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.725+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~4WIO8Z3FZUJV.2 size=926897 2024-03-04T17:50:29.728 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.729+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=4WIO8Z3FZUJV tag=M7ZWR1TY8XUC type=write size=4072625 2024-03-04T17:50:29.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.733+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=6FWJNYQA3UKY type=del size=952091 2024-03-04T17:50:29.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.737+0000 7fda08af98c0 -1 < failed to complete operation key=V1LDB2HXZ7EF tag=ZXY1TEKZDLSQ type=del size=1666245: (22) Invalid argument 2024-03-04T17:50:29.743 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.745+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~DN0LBISDKXKN.0 size=1048576 2024-03-04T17:50:29.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.749+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~DN0LBISDKXKN.1 size=1048576 2024-03-04T17:50:29.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.749+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~DN0LBISDKXKN.2 size=537626 2024-03-04T17:50:29.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.749+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=DN0LBISDKXKN tag=JYJOPIA8FHPL type=write size=3683354 2024-03-04T17:50:29.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.753+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=93GQMJKIRJBW type=del 2024-03-04T17:50:29.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.757+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=2Q5F6J17AM2A type=del size=2549254 2024-03-04T17:50:29.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.765+0000 7fda08af98c0 1 < suggested update operation key=AO3QS6CM7I5D tag=DL52E1QSJOVD type=del size=1797155 2024-03-04T17:50:29.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.769+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=NB45B4LVU9RM type=del size=859495 2024-03-04T17:50:29.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.773+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~AGGBGYGE7ZZ7.0 size=1048576 2024-03-04T17:50:29.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.777+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~AGGBGYGE7ZZ7.1 size=805221 2024-03-04T17:50:29.780 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.781+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=AGGBGYGE7ZZ7 tag=P9VBWM2THRR9 type=write size=2902373 2024-03-04T17:50:29.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.785+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=24EID8XJIWJU type=del size=1935857 2024-03-04T17:50:29.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.789+0000 7fda08af98c0 1 < canceled operation key=JA4JK99I0942 tag=HFJMX04EW7DY type=del size=1163489 2024-03-04T17:50:29.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.793+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=GHVQ2WNVAWXD type=del size=1482937 2024-03-04T17:50:29.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.801+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~2RF8XIRP25Z9.0 size=1048576 2024-03-04T17:50:29.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.801+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~2RF8XIRP25Z9.1 size=1048576 2024-03-04T17:50:29.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.805+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~2RF8XIRP25Z9.2 size=931537 2024-03-04T17:50:29.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.813+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=2RF8XIRP25Z9 tag=LP84OAIFEQA6 type=write size=4077265 2024-03-04T17:50:29.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.817+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=C00VLE8QN9CB type=del size=4106232 2024-03-04T17:50:29.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.821+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~HJK9RYPYKG4J.0 size=1048576 2024-03-04T17:50:29.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.825+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~HJK9RYPYKG4J.1 size=1048576 2024-03-04T17:50:29.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.825+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~HJK9RYPYKG4J.2 size=471294 2024-03-04T17:50:29.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.833+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=HJK9RYPYKG4J tag=EQQH2VK1BRFA type=write size=3617022 2024-03-04T17:50:29.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.837+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=JLAS2IR2VHS6 type=write size=357757 2024-03-04T17:50:29.840 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.841+0000 7fda08af98c0 1 < completed write operation key=HCKJZUMAK0Z0 tag=Q8OU4FQ7MLZ0 type=write size=883302 2024-03-04T17:50:29.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.845+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=MHZVPO2M8WY5 type=del size=3656377 2024-03-04T17:50:29.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.849+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=FEYPEUM25H2R type=del 2024-03-04T17:50:29.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.853+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=92JO7S3U9BEZ type=del size=1378232 2024-03-04T17:50:29.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.857+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~KINXD4WTM9EO.0 size=1048576 2024-03-04T17:50:29.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.861+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~KINXD4WTM9EO.1 size=970265 2024-03-04T17:50:29.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.865+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=KINXD4WTM9EO tag=JSK50KQ3HL7P type=write size=3067417 2024-03-04T17:50:29.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.869+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=NAJCE125H5LT type=del size=4162164 2024-03-04T17:50:29.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.873+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~63VH0NOABB4T.0 size=1048576 2024-03-04T17:50:29.875 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.877+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~63VH0NOABB4T.1 size=987450 2024-03-04T17:50:29.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.881+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=63VH0NOABB4T tag=69W5OFY8OUHY type=write size=3084602 2024-03-04T17:50:29.886 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.889+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=7KJJXG5588IL type=del size=3507001 2024-03-04T17:50:29.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.893+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~D34W27JM14KB.0 size=1048576 2024-03-04T17:50:29.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.897+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~D34W27JM14KB.1 size=224109 2024-03-04T17:50:29.900 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.901+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=D34W27JM14KB tag=O95V0TBEY9BG type=write size=2321261 2024-03-04T17:50:29.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.909+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~K8HZ83T6H5AL.0 size=743721 2024-03-04T17:50:29.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.909+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=K8HZ83T6H5AL tag=ASMBLCKL6W3F type=write size=1792297 2024-03-04T17:50:29.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.913+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=GU10N656201T type=del 2024-03-04T17:50:29.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.917+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=EUGT636ITN98 type=write size=699153 2024-03-04T17:50:29.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.925+0000 7fda08af98c0 1 < suggested update operation key=AO3QS6CM7I5D tag=9554JFH3C2G6 type=del size=1722906 2024-03-04T17:50:29.927 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.929+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=PS0TB4Q4TR2T type=del size=2833720 2024-03-04T17:50:29.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.933+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=S3NSWE5S3K3X type=del 2024-03-04T17:50:29.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.937+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=66BACD5OGVH5 type=del size=2137568 2024-03-04T17:50:29.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.941+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=6FWJNYQA3UKY type=del 2024-03-04T17:50:29.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.945+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=IBUPBD09G3E8 type=del size=2242499 2024-03-04T17:50:29.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.949+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~DN0LBISDKXKN.0 size=1048576 2024-03-04T17:50:29.949 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.949+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~DN0LBISDKXKN.1 size=1048576 2024-03-04T17:50:29.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.953+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~DN0LBISDKXKN.2 size=537626 2024-03-04T17:50:29.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.961+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=DN0LBISDKXKN tag=JYJOPIA8FHPL type=write size=3683354 2024-03-04T17:50:29.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.965+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=HMX55M9GJGUJ type=del size=2976814 2024-03-04T17:50:29.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.969+0000 7fda08af98c0 1 < canceled operation key=750NSVFBS03A tag=2Q5F6J17AM2A type=del size=2549254 2024-03-04T17:50:29.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.973+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=WXIBVJCU3P6A type=del size=606643 2024-03-04T17:50:29.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.977+0000 7fda08af98c0 1 < suggested update operation key=XG5I3SDRE5JM tag=NB45B4LVU9RM type=del size=859495 2024-03-04T17:50:29.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.985+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~9DRMQ4QMKQKS.0 size=1048576 2024-03-04T17:50:29.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.989+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~9DRMQ4QMKQKS.1 size=1048576 2024-03-04T17:50:29.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.993+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~9DRMQ4QMKQKS.2 size=786090 2024-03-04T17:50:29.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.993+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=9DRMQ4QMKQKS tag=NEGWV49ARS8E type=write size=3931818 2024-03-04T17:50:29.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:29.997+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=24EID8XJIWJU type=del 2024-03-04T17:50:30.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.005+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~PSWS1D8FANSV.0 size=114960 2024-03-04T17:50:30.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.005+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=PSWS1D8FANSV tag=FD4FMKAPCQ9T type=write size=1163536 2024-03-04T17:50:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.013+0000 7fda08af98c0 1 < suggested remove operation key=RXVSQVVNXVF0 tag=GHVQ2WNVAWXD type=del 2024-03-04T17:50:30.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.021+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~80G4NLSKU8FC.0 size=126636 2024-03-04T17:50:30.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.021+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=80G4NLSKU8FC tag=HMWH12C2QCSX type=write size=1175212 2024-03-04T17:50:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.025+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=C00VLE8QN9CB type=del 2024-03-04T17:50:30.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.033+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~QTQAAQ2TIVIK.0 size=1048576 2024-03-04T17:50:30.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.033+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~QTQAAQ2TIVIK.1 size=1048576 2024-03-04T17:50:30.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.037+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~QTQAAQ2TIVIK.2 size=764940 2024-03-04T17:50:30.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.037+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=QTQAAQ2TIVIK tag=BV5BKXROOAS8 type=write size=3910668 2024-03-04T17:50:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.041+0000 7fda08af98c0 1 < completed write operation key=DJCCQ6S0E7UG tag=JLAS2IR2VHS6 type=write size=357757 2024-03-04T17:50:30.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.045+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=1CT09QBI9AQS type=del size=2200665 2024-03-04T17:50:30.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.049+0000 7fda08af98c0 1 < canceled operation key=HCKJZUMAK0Z0 tag=MHZVPO2M8WY5 type=del size=3656377 2024-03-04T17:50:30.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.053+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~3FVPTR2CC0X1.0 size=1048576 2024-03-04T17:50:30.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.057+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~3FVPTR2CC0X1.1 size=419973 2024-03-04T17:50:30.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.057+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=3FVPTR2CC0X1 tag=I1GS3F1814LL type=write size=2517125 2024-03-04T17:50:30.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.061+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=92JO7S3U9BEZ type=del 2024-03-04T17:50:30.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.065+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=6IY4Q39IDJII type=write size=793747 2024-03-04T17:50:30.067 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.069+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=NAJCE125H5LT type=del 2024-03-04T17:50:30.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.073+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=0ZTWONQ68AEF type=del size=2172329 2024-03-04T17:50:30.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.077+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=7KJJXG5588IL type=del 2024-03-04T17:50:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.081+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~8HAT87ADKWOJ.0 size=1048576 2024-03-04T17:50:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.085+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~8HAT87ADKWOJ.1 size=1048576 2024-03-04T17:50:30.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.085+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~8HAT87ADKWOJ.2 size=990780 2024-03-04T17:50:30.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.085+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=8HAT87ADKWOJ tag=TROZFGV29YNV type=write size=4136508 2024-03-04T17:50:30.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.089+0000 7fda08af98c0 1 < canceled part key=_multipart_BYQFZSJ3WH07.2~K8HZ83T6H5AL.0 size=743721 2024-03-04T17:50:30.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.093+0000 7fda08af98c0 1 < canceled multipart upload key=BYQFZSJ3WH07 upload=K8HZ83T6H5AL tag=ASMBLCKL6W3F type=write size=1792297 2024-03-04T17:50:30.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.097+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=39DVT0U2IB48 type=del size=3899854 2024-03-04T17:50:30.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.101+0000 7fda08af98c0 1 < completed write operation key=HCKJZUMAK0Z0 tag=EUGT636ITN98 type=write size=699153 2024-03-04T17:50:30.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.105+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=QRSQOWALN85V type=del size=1926168 2024-03-04T17:50:30.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.113+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=PS0TB4Q4TR2T type=del 2024-03-04T17:50:30.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.117+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=C0NY732RD5B6 type=del size=352719 2024-03-04T17:50:30.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.121+0000 7fda08af98c0 -1 < failed to complete operation key=RXVSQVVNXVF0 tag=66BACD5OGVH5 type=del size=2137568: (22) Invalid argument 2024-03-04T17:50:30.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.125+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=76QMYWVC8CWO type=del size=474934 2024-03-04T17:50:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.129+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=IBUPBD09G3E8 type=del 2024-03-04T17:50:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.133+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=BTMPEZEFKRSC type=del size=1710969 2024-03-04T17:50:30.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.137+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=HMX55M9GJGUJ type=del 2024-03-04T17:50:30.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.141+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=6FHU0KLAXPNR type=del size=2670897 2024-03-04T17:50:30.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.145+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=WXIBVJCU3P6A type=del 2024-03-04T17:50:30.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.149+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=UU1HRCCLH17P type=write size=718852 2024-03-04T17:50:30.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.153+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~9DRMQ4QMKQKS.0 size=1048576 2024-03-04T17:50:30.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.153+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~9DRMQ4QMKQKS.1 size=1048576 2024-03-04T17:50:30.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.153+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~9DRMQ4QMKQKS.2 size=786090 2024-03-04T17:50:30.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.161+0000 7fda08af98c0 1 < canceled multipart upload key=JA4JK99I0942 upload=9DRMQ4QMKQKS tag=NEGWV49ARS8E type=write size=3931818 2024-03-04T17:50:30.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.169+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~8ZYZ4AY5VZZ9.0 size=123186 2024-03-04T17:50:30.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.169+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=8ZYZ4AY5VZZ9 tag=3X427QJG3589 type=write size=1171762 2024-03-04T17:50:30.170 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.173+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~PSWS1D8FANSV.0 size=114960 2024-03-04T17:50:30.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.177+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=PSWS1D8FANSV tag=FD4FMKAPCQ9T type=write size=1163536 2024-03-04T17:50:30.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.185+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~0Z4665S32Z9B.0 size=1048576 2024-03-04T17:50:30.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.189+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~0Z4665S32Z9B.1 size=980347 2024-03-04T17:50:30.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.189+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=0Z4665S32Z9B tag=NLD07PTOFG70 type=write size=3077499 2024-03-04T17:50:30.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.193+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~80G4NLSKU8FC.0 size=126636 2024-03-04T17:50:30.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.201+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=80G4NLSKU8FC tag=HMWH12C2QCSX type=write size=1175212 2024-03-04T17:50:30.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.205+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=611WRL7VDM07 type=write size=644253 2024-03-04T17:50:30.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.209+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~QTQAAQ2TIVIK.0 size=1048576 2024-03-04T17:50:30.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.217+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~QTQAAQ2TIVIK.1 size=1048576 2024-03-04T17:50:30.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.221+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~QTQAAQ2TIVIK.2 size=764940 2024-03-04T17:50:30.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.225+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=QTQAAQ2TIVIK tag=BV5BKXROOAS8 type=write size=3910668 2024-03-04T17:50:30.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.229+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~59FH2X0JQLF7.0 size=1048576 2024-03-04T17:50:30.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.233+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~59FH2X0JQLF7.1 size=354446 2024-03-04T17:50:30.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.233+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=59FH2X0JQLF7 tag=B567Y96RK8XG type=write size=2451598 2024-03-04T17:50:30.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.237+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=1CT09QBI9AQS type=del 2024-03-04T17:50:30.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.237+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=UH40GOJK6R0J type=del size=3053979 2024-03-04T17:50:30.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.241+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~3FVPTR2CC0X1.0 size=1048576 2024-03-04T17:50:30.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.245+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~3FVPTR2CC0X1.1 size=419973 2024-03-04T17:50:30.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.249+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=3FVPTR2CC0X1 tag=I1GS3F1814LL type=write size=2517125 2024-03-04T17:50:30.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.257+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~6GX77PCPSMKL.0 size=1048576 2024-03-04T17:50:30.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.261+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~6GX77PCPSMKL.1 size=1048576 2024-03-04T17:50:30.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.261+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~6GX77PCPSMKL.2 size=107371 2024-03-04T17:50:30.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.261+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=6GX77PCPSMKL tag=G5XVGOUYK6TL type=write size=3253099 2024-03-04T17:50:30.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.265+0000 7fda08af98c0 1 < canceled operation key=8YL7C9PI58TK tag=6IY4Q39IDJII type=write size=793747 2024-03-04T17:50:30.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.273+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~F9Y78R0KOKJD.0 size=1048576 2024-03-04T17:50:30.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.273+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~F9Y78R0KOKJD.1 size=1048576 2024-03-04T17:50:30.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.277+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~F9Y78R0KOKJD.2 size=552303 2024-03-04T17:50:30.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.277+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=F9Y78R0KOKJD tag=14ECEYINJB7W type=write size=3698031 2024-03-04T17:50:30.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.281+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=0ZTWONQ68AEF type=del 2024-03-04T17:50:30.287 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.289+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~3U3L8JJPN4HA.0 size=475540 2024-03-04T17:50:30.287 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.289+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=3U3L8JJPN4HA tag=79WCTFFTNFEQ type=write size=1524116 2024-03-04T17:50:30.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.293+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~8HAT87ADKWOJ.0 size=1048576 2024-03-04T17:50:30.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.293+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~8HAT87ADKWOJ.1 size=1048576 2024-03-04T17:50:30.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.297+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~8HAT87ADKWOJ.2 size=990780 2024-03-04T17:50:30.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.305+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=8HAT87ADKWOJ tag=TROZFGV29YNV type=write size=4136508 2024-03-04T17:50:30.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.309+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=UY7HGQDN7Q0M type=del size=1323236 2024-03-04T17:50:30.310 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.313+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=39DVT0U2IB48 type=del 2024-03-04T17:50:30.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.317+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=OGSMKAWE2F3Y type=del size=503442 2024-03-04T17:50:30.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.321+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=QRSQOWALN85V type=del 2024-03-04T17:50:30.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.321+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=X3BQFLX09XZ8 type=del size=1931589 2024-03-04T17:50:30.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.329+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=C0NY732RD5B6 type=del 2024-03-04T17:50:30.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.329+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=EDQQKHXIHELW type=del size=2597539 2024-03-04T17:50:30.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.337+0000 7fda08af98c0 1 < suggested remove operation key=3UWSL7NJLACY tag=76QMYWVC8CWO type=del 2024-03-04T17:50:30.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.341+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=I3ELLL0OKKYI type=del size=2188028 2024-03-04T17:50:30.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.345+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=BTMPEZEFKRSC type=del 2024-03-04T17:50:30.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.357+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~NB5OAAGSKQ9A.0 size=1048576 2024-03-04T17:50:30.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.361+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~NB5OAAGSKQ9A.1 size=1048576 2024-03-04T17:50:30.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.361+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~NB5OAAGSKQ9A.2 size=345348 2024-03-04T17:50:30.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.361+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=NB5OAAGSKQ9A tag=2NJR03L4LJTE type=write size=3491076 2024-03-04T17:50:30.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.365+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=6FHU0KLAXPNR type=del 2024-03-04T17:50:30.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.373+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=TIV0OL85X39X type=del size=4028369 2024-03-04T17:50:30.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.377+0000 7fda08af98c0 -1 < failed to complete operation key=3UWSL7NJLACY tag=UU1HRCCLH17P type=write size=718852: (22) Invalid argument 2024-03-04T17:50:30.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.377+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=G88ZUSPGJJLT type=del size=3419583 2024-03-04T17:50:30.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.381+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~8ZYZ4AY5VZZ9.0 size=123186 2024-03-04T17:50:30.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.389+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=8ZYZ4AY5VZZ9 tag=3X427QJG3589 type=write size=1171762 2024-03-04T17:50:30.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.393+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=HQVWWE8KI0HU type=del size=2166531 2024-03-04T17:50:30.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.397+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~0Z4665S32Z9B.0 size=1048576 2024-03-04T17:50:30.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.401+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~0Z4665S32Z9B.1 size=980347 2024-03-04T17:50:30.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.405+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=0Z4665S32Z9B tag=NLD07PTOFG70 type=write size=3077499 2024-03-04T17:50:30.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.413+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~IDLWJY6KBI5O.0 size=300927 2024-03-04T17:50:30.412 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.413+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=IDLWJY6KBI5O tag=W2F7OC1X15ZS type=write size=1349503 2024-03-04T17:50:30.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.417+0000 7fda08af98c0 1 < canceled operation key=GDGTJZ21TG32 tag=611WRL7VDM07 type=write size=644253 2024-03-04T17:50:30.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.421+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=GB5NHT88RLTS type=del size=1787055 2024-03-04T17:50:30.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.425+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~59FH2X0JQLF7.0 size=1048576 2024-03-04T17:50:30.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.429+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~59FH2X0JQLF7.1 size=354446 2024-03-04T17:50:30.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.433+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=59FH2X0JQLF7 tag=B567Y96RK8XG type=write size=2451598 2024-03-04T17:50:30.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.437+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=G1K4M3CW28CJ type=write size=64772 2024-03-04T17:50:30.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.441+0000 7fda08af98c0 1 < canceled operation key=09KBCA2XURTF tag=UH40GOJK6R0J type=del size=3053979 2024-03-04T17:50:30.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.449+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~OG7MSTOM5L03.0 size=406825 2024-03-04T17:50:30.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.449+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=OG7MSTOM5L03 tag=DPU9JV8OS3A3 type=write size=1455401 2024-03-04T17:50:30.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.453+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~6GX77PCPSMKL.0 size=1048576 2024-03-04T17:50:30.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.457+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~6GX77PCPSMKL.1 size=1048576 2024-03-04T17:50:30.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.457+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~6GX77PCPSMKL.2 size=107371 2024-03-04T17:50:30.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.465+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=6GX77PCPSMKL tag=G5XVGOUYK6TL type=write size=3253099 2024-03-04T17:50:30.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.469+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=JQEYSH31APOX type=del size=2246981 2024-03-04T17:50:30.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.473+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~F9Y78R0KOKJD.0 size=1048576 2024-03-04T17:50:30.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.473+0000 7fda08af98c0 1 < canceled part key=_multipart_ITIVF3UAT9NK.2~F9Y78R0KOKJD.1 size=1048576 2024-03-04T17:50:30.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.477+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~F9Y78R0KOKJD.2 size=552303 2024-03-04T17:50:30.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.481+0000 7fda08af98c0 1 < canceled multipart upload key=ITIVF3UAT9NK upload=F9Y78R0KOKJD tag=14ECEYINJB7W type=write size=3698031 2024-03-04T17:50:30.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.485+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=29WY0THXHGVZ type=write size=94144 2024-03-04T17:50:30.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.489+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~3U3L8JJPN4HA.0 size=475540 2024-03-04T17:50:30.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.497+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=3U3L8JJPN4HA tag=79WCTFFTNFEQ type=write size=1524116 2024-03-04T17:50:30.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.501+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=MIYHN97ZSE59 type=del size=265782 2024-03-04T17:50:30.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.505+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=UY7HGQDN7Q0M type=del 2024-03-04T17:50:30.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.505+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=6P8NUP5TH6AD type=del size=361195 2024-03-04T17:50:30.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.513+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=OGSMKAWE2F3Y type=del 2024-03-04T17:50:30.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.517+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=MER0HF1QWF9S type=del size=2439727 2024-03-04T17:50:30.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.525+0000 7fda08af98c0 1 < suggested update operation key=8YL7C9PI58TK tag=X3BQFLX09XZ8 type=del size=1931589 2024-03-04T17:50:30.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.529+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=IIPDNNZL7466 type=del size=1816961 2024-03-04T17:50:30.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.533+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=EDQQKHXIHELW type=del 2024-03-04T17:50:30.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.537+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~QTK8G2G65FYY.0 size=1048576 2024-03-04T17:50:30.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.541+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~QTK8G2G65FYY.1 size=1048576 2024-03-04T17:50:30.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.545+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~QTK8G2G65FYY.2 size=733146 2024-03-04T17:50:30.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.545+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=QTK8G2G65FYY tag=7I6VRWRXZC9W type=write size=3878874 2024-03-04T17:50:30.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.549+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=I3ELLL0OKKYI type=del 2024-03-04T17:50:30.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.553+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~SPQ92PHX06S9.0 size=1048576 2024-03-04T17:50:30.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.557+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~SPQ92PHX06S9.1 size=1048576 2024-03-04T17:50:30.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.561+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~SPQ92PHX06S9.2 size=948724 2024-03-04T17:50:30.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.561+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=SPQ92PHX06S9 tag=73DGU20JVOKR type=write size=4094452 2024-03-04T17:50:30.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.565+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~NB5OAAGSKQ9A.0 size=1048576 2024-03-04T17:50:30.565 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.565+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~NB5OAAGSKQ9A.1 size=1048576 2024-03-04T17:50:30.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.569+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~NB5OAAGSKQ9A.2 size=345348 2024-03-04T17:50:30.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.577+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=NB5OAAGSKQ9A tag=2NJR03L4LJTE type=write size=3491076 2024-03-04T17:50:30.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.581+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~FBWJKY0CCLYT.0 size=1048576 2024-03-04T17:50:30.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.585+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~FBWJKY0CCLYT.1 size=1048576 2024-03-04T17:50:30.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.589+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~FBWJKY0CCLYT.2 size=624898 2024-03-04T17:50:30.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.589+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=FBWJKY0CCLYT tag=D5YCBP542SZ2 type=write size=3770626 2024-03-04T17:50:30.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.593+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=TIV0OL85X39X type=del 2024-03-04T17:50:30.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.597+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=6LMM5LR005AK type=del size=459827 2024-03-04T17:50:30.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.601+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=G88ZUSPGJJLT type=del 2024-03-04T17:50:30.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.605+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=3KCR44OBKYSK type=write size=125265 2024-03-04T17:50:30.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.609+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=HQVWWE8KI0HU type=del 2024-03-04T17:50:30.615 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.617+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~BPWWUZPVZ1XR.0 size=1048576 2024-03-04T17:50:30.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.621+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~BPWWUZPVZ1XR.1 size=954990 2024-03-04T17:50:30.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.621+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=BPWWUZPVZ1XR tag=LOET2BR89Z0D type=write size=3052142 2024-03-04T17:50:30.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.625+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~IDLWJY6KBI5O.0 size=300927 2024-03-04T17:50:30.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.633+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=IDLWJY6KBI5O tag=W2F7OC1X15ZS type=write size=1349503 2024-03-04T17:50:30.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.637+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~8MNGV8MKX1NV.0 size=976762 2024-03-04T17:50:30.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.637+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=8MNGV8MKX1NV tag=LN9ZQXB219E1 type=write size=2025338 2024-03-04T17:50:30.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.641+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=GB5NHT88RLTS type=del 2024-03-04T17:50:30.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.649+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~NN7KME685IYY.0 size=1048576 2024-03-04T17:50:30.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.653+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~NN7KME685IYY.1 size=1048576 2024-03-04T17:50:30.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.653+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~NN7KME685IYY.2 size=527950 2024-03-04T17:50:30.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.653+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=NN7KME685IYY tag=J0YEFO457ROX type=write size=3673678 2024-03-04T17:50:30.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.661+0000 7fda08af98c0 1 < canceled operation key=1OZKPHMPV3OQ tag=G1K4M3CW28CJ type=write size=64772 2024-03-04T17:50:30.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.665+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=AQKK24NOYHV8 type=del size=4012045 2024-03-04T17:50:30.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.669+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~OG7MSTOM5L03.0 size=406825 2024-03-04T17:50:30.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.673+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=OG7MSTOM5L03 tag=DPU9JV8OS3A3 type=write size=1455401 2024-03-04T17:50:30.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.677+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=CP03S1SE0BZS type=del size=2317491 2024-03-04T17:50:30.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.685+0000 7fda08af98c0 1 < suggested remove operation key=RXVSQVVNXVF0 tag=JQEYSH31APOX type=del 2024-03-04T17:50:30.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.689+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=ER8OW8MZXXA0 type=del size=773248 2024-03-04T17:50:30.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.693+0000 7fda08af98c0 1 < completed write operation key=X5MEJ7G8X9HE tag=29WY0THXHGVZ type=write size=94144 2024-03-04T17:50:30.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.697+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=9JMG4KG66HD5 type=del size=2123370 2024-03-04T17:50:30.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.701+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=MIYHN97ZSE59 type=del 2024-03-04T17:50:30.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.709+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~XJGNUBZT02R0.0 size=1048576 2024-03-04T17:50:30.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.713+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~XJGNUBZT02R0.1 size=1048576 2024-03-04T17:50:30.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.717+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~XJGNUBZT02R0.2 size=603449 2024-03-04T17:50:30.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.717+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=XJGNUBZT02R0 tag=F4MCQMTVU5KU type=write size=3749177 2024-03-04T17:50:30.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.721+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=6P8NUP5TH6AD type=del 2024-03-04T17:50:30.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.725+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=3RQNLAO5P9RC type=del size=2686767 2024-03-04T17:50:30.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.729+0000 7fda08af98c0 -1 < no bi entry to suggest for operation key=RXVSQVVNXVF0 tag=MER0HF1QWF9S type=del size=2439727: (2) No such file or directory 2024-03-04T17:50:30.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.737+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~D7J2V06IT0QC.0 size=1048576 2024-03-04T17:50:30.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.737+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~D7J2V06IT0QC.1 size=1048576 2024-03-04T17:50:30.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.741+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~D7J2V06IT0QC.2 size=1004877 2024-03-04T17:50:30.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.741+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=D7J2V06IT0QC tag=RN34ODCP52S4 type=write size=4150605 2024-03-04T17:50:30.744 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.745+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=IIPDNNZL7466 type=del 2024-03-04T17:50:30.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.749+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=N5F1E15SM913 type=del size=2703434 2024-03-04T17:50:30.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.753+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~QTK8G2G65FYY.0 size=1048576 2024-03-04T17:50:30.755 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.757+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~QTK8G2G65FYY.1 size=1048576 2024-03-04T17:50:30.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.761+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~QTK8G2G65FYY.2 size=733146 2024-03-04T17:50:30.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.765+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=QTK8G2G65FYY tag=7I6VRWRXZC9W type=write size=3878874 2024-03-04T17:50:30.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.769+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=RIV2W4DEVKCA type=del size=2298660 2024-03-04T17:50:30.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.773+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~SPQ92PHX06S9.0 size=1048576 2024-03-04T17:50:30.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.777+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~SPQ92PHX06S9.1 size=1048576 2024-03-04T17:50:30.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.781+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~SPQ92PHX06S9.2 size=948724 2024-03-04T17:50:30.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.785+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=SPQ92PHX06S9 tag=73DGU20JVOKR type=write size=4094452 2024-03-04T17:50:30.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.793+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~Q18DRSMTUG7Q.0 size=1048576 2024-03-04T17:50:30.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.797+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~Q18DRSMTUG7Q.1 size=999561 2024-03-04T17:50:30.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.797+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=Q18DRSMTUG7Q tag=803HYSLC67IW type=write size=3096713 2024-03-04T17:50:30.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.801+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~FBWJKY0CCLYT.0 size=1048576 2024-03-04T17:50:30.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.801+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~FBWJKY0CCLYT.1 size=1048576 2024-03-04T17:50:30.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.805+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~FBWJKY0CCLYT.2 size=624898 2024-03-04T17:50:30.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.809+0000 7fda08af98c0 1 < canceled multipart upload key=JA4JK99I0942 upload=FBWJKY0CCLYT tag=D5YCBP542SZ2 type=write size=3770626 2024-03-04T17:50:30.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.817+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~JUSMWUQZZ7EO.0 size=1048576 2024-03-04T17:50:30.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.821+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~JUSMWUQZZ7EO.1 size=1048576 2024-03-04T17:50:30.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.821+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~JUSMWUQZZ7EO.2 size=436319 2024-03-04T17:50:30.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.821+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=JUSMWUQZZ7EO tag=M6BNQYYT0GXM type=write size=3582047 2024-03-04T17:50:30.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.825+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=6LMM5LR005AK type=del 2024-03-04T17:50:30.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.833+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~5552VBYO37XN.0 size=180562 2024-03-04T17:50:30.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.833+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=5552VBYO37XN tag=661Y6553I1FW type=write size=1229138 2024-03-04T17:50:30.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.837+0000 7fda08af98c0 1 < completed write operation key=3UWSL7NJLACY tag=3KCR44OBKYSK type=write size=125265 2024-03-04T17:50:30.840 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.841+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=J4XCVQMRVY8H type=write size=76168 2024-03-04T17:50:30.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.845+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~BPWWUZPVZ1XR.0 size=1048576 2024-03-04T17:50:30.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.849+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~BPWWUZPVZ1XR.1 size=954990 2024-03-04T17:50:30.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.853+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=BPWWUZPVZ1XR tag=LOET2BR89Z0D type=write size=3052142 2024-03-04T17:50:30.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.857+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=BKA0K42FAG1V type=write size=1043154 2024-03-04T17:50:30.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.861+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~8MNGV8MKX1NV.0 size=976762 2024-03-04T17:50:30.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.869+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=8MNGV8MKX1NV tag=LN9ZQXB219E1 type=write size=2025338 2024-03-04T17:50:30.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.873+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~1J4V8JYFK6H5.0 size=277455 2024-03-04T17:50:30.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.873+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=1J4V8JYFK6H5 tag=RR12331LVCJM type=write size=1326031 2024-03-04T17:50:30.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.877+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~NN7KME685IYY.0 size=1048576 2024-03-04T17:50:30.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.881+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~NN7KME685IYY.1 size=1048576 2024-03-04T17:50:30.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.885+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~NN7KME685IYY.2 size=527950 2024-03-04T17:50:30.906 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.905+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=NN7KME685IYY tag=J0YEFO457ROX type=write size=3673678 2024-03-04T17:50:30.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.913+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~MLTGU0YT4LD4.0 size=1048576 2024-03-04T17:50:30.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.917+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~MLTGU0YT4LD4.1 size=1048576 2024-03-04T17:50:30.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.921+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~MLTGU0YT4LD4.2 size=1006441 2024-03-04T17:50:30.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.921+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=MLTGU0YT4LD4 tag=T72BDRDR8IQC type=write size=4152169 2024-03-04T17:50:30.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.925+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=AQKK24NOYHV8 type=del 2024-03-04T17:50:30.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.929+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=TXZ8NY9OZI9X type=del size=3576612 2024-03-04T17:50:30.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.933+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=CP03S1SE0BZS type=del 2024-03-04T17:50:30.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.937+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=EXCD3JPNZC6W type=del size=610053 2024-03-04T17:50:30.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.941+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=ER8OW8MZXXA0 type=del 2024-03-04T17:50:30.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.945+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~HAI51D4K6EON.0 size=415676 2024-03-04T17:50:30.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.945+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=HAI51D4K6EON tag=TMFCL6HJ32PU type=write size=1464252 2024-03-04T17:50:30.949 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.949+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=9JMG4KG66HD5 type=del 2024-03-04T17:50:30.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.957+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~S0FJT6LN4Z8S.0 size=1048576 2024-03-04T17:50:30.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.961+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~S0FJT6LN4Z8S.1 size=1048576 2024-03-04T17:50:30.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.965+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~S0FJT6LN4Z8S.2 size=298960 2024-03-04T17:50:30.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.965+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=S0FJT6LN4Z8S tag=BVEGIIWT5W1Z type=write size=3444688 2024-03-04T17:50:30.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.969+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~XJGNUBZT02R0.0 size=1048576 2024-03-04T17:50:30.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.973+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~XJGNUBZT02R0.1 size=1048576 2024-03-04T17:50:30.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.977+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~XJGNUBZT02R0.2 size=603449 2024-03-04T17:50:30.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.981+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=XJGNUBZT02R0 tag=F4MCQMTVU5KU type=write size=3749177 2024-03-04T17:50:30.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.985+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=BQXROBVNG30Q type=del size=2987741 2024-03-04T17:50:30.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.989+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=3RQNLAO5P9RC type=del 2024-03-04T17:50:30.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.993+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=9O5G22OAGGIB type=del size=2642827 2024-03-04T17:50:30.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:30.997+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~D7J2V06IT0QC.0 size=1048576 2024-03-04T17:50:30.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.001+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~D7J2V06IT0QC.1 size=1048576 2024-03-04T17:50:31.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.001+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~D7J2V06IT0QC.2 size=1004877 2024-03-04T17:50:31.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.009+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=D7J2V06IT0QC tag=RN34ODCP52S4 type=write size=4150605 2024-03-04T17:50:31.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.013+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=83L4JCL51VZH type=write size=696344 2024-03-04T17:50:31.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.017+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=N5F1E15SM913 type=del 2024-03-04T17:50:31.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.021+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=6ZFVH548I6Y8 type=del size=2641436 2024-03-04T17:50:31.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.025+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=RIV2W4DEVKCA type=del 2024-03-04T17:50:31.027 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.029+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=9RBAJI0LJ5GZ type=del size=2275197 2024-03-04T17:50:31.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.033+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~Q18DRSMTUG7Q.0 size=1048576 2024-03-04T17:50:31.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.033+0000 7fda08af98c0 1 < canceled part key=_multipart_C4B45GL40IMZ.2~Q18DRSMTUG7Q.1 size=999561 2024-03-04T17:50:31.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.037+0000 7fda08af98c0 1 < canceled multipart upload key=C4B45GL40IMZ upload=Q18DRSMTUG7Q tag=803HYSLC67IW type=write size=3096713 2024-03-04T17:50:31.040 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.041+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=Z9D35DZA2K9T type=del size=3741626 2024-03-04T17:50:31.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.045+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~JUSMWUQZZ7EO.0 size=1048576 2024-03-04T17:50:31.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.049+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~JUSMWUQZZ7EO.1 size=1048576 2024-03-04T17:50:31.051 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.053+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~JUSMWUQZZ7EO.2 size=436319 2024-03-04T17:50:31.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.061+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=JUSMWUQZZ7EO tag=M6BNQYYT0GXM type=write size=3582047 2024-03-04T17:50:31.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.069+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~MJ4OPXEKSV1G.0 size=1048576 2024-03-04T17:50:31.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.069+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~MJ4OPXEKSV1G.1 size=1048576 2024-03-04T17:50:31.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.073+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~MJ4OPXEKSV1G.2 size=257389 2024-03-04T17:50:31.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.073+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=MJ4OPXEKSV1G tag=3NXBR8ICUHZG type=write size=3403117 2024-03-04T17:50:31.077 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.077+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~5552VBYO37XN.0 size=180562 2024-03-04T17:50:31.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.085+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=5552VBYO37XN tag=661Y6553I1FW type=write size=1229138 2024-03-04T17:50:31.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.093+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=3KR6639V0Z7O type=del size=3149530 2024-03-04T17:50:31.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.093+0000 7fda08af98c0 1 < completed write operation key=M5BMX7B7RZYL tag=J4XCVQMRVY8H type=write size=76168 2024-03-04T17:50:31.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.101+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~1XN20BHS52LO.0 size=1048576 2024-03-04T17:50:31.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.105+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~1XN20BHS52LO.1 size=1048576 2024-03-04T17:50:31.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.109+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~1XN20BHS52LO.2 size=639404 2024-03-04T17:50:31.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.109+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=1XN20BHS52LO tag=F6UA33XXNWLV type=write size=3785132 2024-03-04T17:50:31.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.113+0000 7fda08af98c0 1 < completed write operation key=HCKJZUMAK0Z0 tag=BKA0K42FAG1V type=write size=1043154 2024-03-04T17:50:31.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.117+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=VCXLPW134FCM type=write size=165147 2024-03-04T17:50:31.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.121+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~1J4V8JYFK6H5.0 size=277455 2024-03-04T17:50:31.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.125+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=1J4V8JYFK6H5 tag=RR12331LVCJM type=write size=1326031 2024-03-04T17:50:31.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.129+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=8GNIBPTI7C4M type=del size=698874 2024-03-04T17:50:31.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.133+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~MLTGU0YT4LD4.0 size=1048576 2024-03-04T17:50:31.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.137+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~MLTGU0YT4LD4.1 size=1048576 2024-03-04T17:50:31.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.141+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~MLTGU0YT4LD4.2 size=1006441 2024-03-04T17:50:31.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.145+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=MLTGU0YT4LD4 tag=T72BDRDR8IQC type=write size=4152169 2024-03-04T17:50:31.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.153+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~9KH8XAHQUTHG.0 size=1048576 2024-03-04T17:50:31.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.157+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~9KH8XAHQUTHG.1 size=1048576 2024-03-04T17:50:31.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.157+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~9KH8XAHQUTHG.2 size=245569 2024-03-04T17:50:31.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.157+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=9KH8XAHQUTHG tag=C7LV94AZKAM4 type=write size=3391297 2024-03-04T17:50:31.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.161+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=TXZ8NY9OZI9X type=del 2024-03-04T17:50:31.168 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.169+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~3MT2O68VAT0L.0 size=1048576 2024-03-04T17:50:31.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.173+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~3MT2O68VAT0L.1 size=1048576 2024-03-04T17:50:31.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.177+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~3MT2O68VAT0L.2 size=825726 2024-03-04T17:50:31.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.177+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=3MT2O68VAT0L tag=NNMGHNYKMLRM type=write size=3971454 2024-03-04T17:50:31.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.181+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=EXCD3JPNZC6W type=del 2024-03-04T17:50:31.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.189+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~WKP6ERV5QYS0.0 size=1048576 2024-03-04T17:50:31.189 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.189+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~WKP6ERV5QYS0.1 size=920443 2024-03-04T17:50:31.189 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.189+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=WKP6ERV5QYS0 tag=LL3IOH0K1OWN type=write size=3017595 2024-03-04T17:50:31.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.193+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~HAI51D4K6EON.0 size=415676 2024-03-04T17:50:31.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.201+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=HAI51D4K6EON tag=TMFCL6HJ32PU type=write size=1464252 2024-03-04T17:50:31.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.209+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~CG1DI53E0YML.0 size=1048576 2024-03-04T17:50:31.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.213+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~CG1DI53E0YML.1 size=785829 2024-03-04T17:50:31.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.213+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=CG1DI53E0YML tag=1LTWPF949EM9 type=write size=2882981 2024-03-04T17:50:31.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.217+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~S0FJT6LN4Z8S.0 size=1048576 2024-03-04T17:50:31.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.221+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~S0FJT6LN4Z8S.1 size=1048576 2024-03-04T17:50:31.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.225+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~S0FJT6LN4Z8S.2 size=298960 2024-03-04T17:50:31.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.229+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=S0FJT6LN4Z8S tag=BVEGIIWT5W1Z type=write size=3444688 2024-03-04T17:50:31.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.233+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=M2K3Q01D1MNB type=del size=3820943 2024-03-04T17:50:31.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.237+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=BQXROBVNG30Q type=del 2024-03-04T17:50:31.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.245+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=0W1O049UAPPS type=write size=595775 2024-03-04T17:50:31.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.249+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=9O5G22OAGGIB type=del 2024-03-04T17:50:31.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.253+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=MNBXGOXC8ZYM type=del size=2026065 2024-03-04T17:50:31.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.257+0000 7fda08af98c0 1 < completed write operation key=HCA7QLGM2O4O tag=83L4JCL51VZH type=write size=696344 2024-03-04T17:50:31.262 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.265+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~X3HZW5EDDNGK.0 size=1048576 2024-03-04T17:50:31.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.269+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~X3HZW5EDDNGK.1 size=840512 2024-03-04T17:50:31.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.269+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=X3HZW5EDDNGK tag=TTUW7ZP0I0OW type=write size=2937664 2024-03-04T17:50:31.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.273+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=6ZFVH548I6Y8 type=del 2024-03-04T17:50:31.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.281+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~D3GVLUUZJ4ZX.0 size=1048576 2024-03-04T17:50:31.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.281+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~D3GVLUUZJ4ZX.1 size=1048576 2024-03-04T17:50:31.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.285+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~D3GVLUUZJ4ZX.2 size=569592 2024-03-04T17:50:31.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.285+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=D3GVLUUZJ4ZX tag=MBCOU0RX8K76 type=write size=3715320 2024-03-04T17:50:31.288 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.289+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=9RBAJI0LJ5GZ type=del 2024-03-04T17:50:31.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.297+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~YETO0LGDB1LE.0 size=1048576 2024-03-04T17:50:31.299 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.301+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~YETO0LGDB1LE.1 size=1048576 2024-03-04T17:50:31.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.305+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~YETO0LGDB1LE.2 size=202631 2024-03-04T17:50:31.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.305+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=YETO0LGDB1LE tag=160UCU3PS2UM type=write size=3348359 2024-03-04T17:50:31.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.309+0000 7fda08af98c0 1 < canceled operation key=IJSYAVR5SWPS tag=Z9D35DZA2K9T type=del size=3741626 2024-03-04T17:50:31.310 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.313+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=4VKNQYZYBNGO type=write size=702350 2024-03-04T17:50:31.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.313+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~MJ4OPXEKSV1G.0 size=1048576 2024-03-04T17:50:31.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.317+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~MJ4OPXEKSV1G.1 size=1048576 2024-03-04T17:50:31.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.321+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~MJ4OPXEKSV1G.2 size=257389 2024-03-04T17:50:31.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.329+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=MJ4OPXEKSV1G tag=3NXBR8ICUHZG type=write size=3403117 2024-03-04T17:50:31.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.333+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=56ZNN0JTNQDK type=write size=809158 2024-03-04T17:50:31.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.337+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=3KR6639V0Z7O type=del size=3149530 2024-03-04T17:50:31.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.341+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=LFJHA91UJ11N type=del size=1566844 2024-03-04T17:50:31.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.345+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~1XN20BHS52LO.0 size=1048576 2024-03-04T17:50:31.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.349+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~1XN20BHS52LO.1 size=1048576 2024-03-04T17:50:31.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.353+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~1XN20BHS52LO.2 size=639404 2024-03-04T17:50:31.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.357+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=1XN20BHS52LO tag=F6UA33XXNWLV type=write size=3785132 2024-03-04T17:50:31.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.365+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=4V04WGU7IDUN type=del size=2786833 2024-03-04T17:50:31.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.369+0000 7fda08af98c0 1 < completed write operation key=750NSVFBS03A tag=VCXLPW134FCM type=write size=165147 2024-03-04T17:50:31.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.373+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=5BO5Z3YWABQE type=del size=744222 2024-03-04T17:50:31.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.377+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=8GNIBPTI7C4M type=del 2024-03-04T17:50:31.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.381+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=03B2HEB0VXAA type=del size=3299756 2024-03-04T17:50:31.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.385+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~9KH8XAHQUTHG.0 size=1048576 2024-03-04T17:50:31.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.389+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~9KH8XAHQUTHG.1 size=1048576 2024-03-04T17:50:31.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.393+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~9KH8XAHQUTHG.2 size=245569 2024-03-04T17:50:31.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.401+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=9KH8XAHQUTHG tag=C7LV94AZKAM4 type=write size=3391297 2024-03-04T17:50:31.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.405+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=RHJFJYCW5K2C type=write size=1004884 2024-03-04T17:50:31.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.409+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~3MT2O68VAT0L.0 size=1048576 2024-03-04T17:50:31.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.409+0000 7fda08af98c0 1 < canceled part key=_multipart_3UWSL7NJLACY.2~3MT2O68VAT0L.1 size=1048576 2024-03-04T17:50:31.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.413+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~3MT2O68VAT0L.2 size=825726 2024-03-04T17:50:31.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.417+0000 7fda08af98c0 1 < canceled multipart upload key=3UWSL7NJLACY upload=3MT2O68VAT0L tag=NNMGHNYKMLRM type=write size=3971454 2024-03-04T17:50:31.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.425+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~AS4ISH14RBUU.0 size=1048576 2024-03-04T17:50:31.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.429+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~AS4ISH14RBUU.1 size=1048576 2024-03-04T17:50:31.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.429+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~AS4ISH14RBUU.2 size=778357 2024-03-04T17:50:31.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.429+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=AS4ISH14RBUU tag=27FZ985T5HBA type=write size=3924085 2024-03-04T17:50:31.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.433+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~WKP6ERV5QYS0.0 size=1048576 2024-03-04T17:50:31.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.437+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~WKP6ERV5QYS0.1 size=920443 2024-03-04T17:50:31.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.445+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=WKP6ERV5QYS0 tag=LL3IOH0K1OWN type=write size=3017595 2024-03-04T17:50:31.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.449+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=0XOAOAF9RVUL type=del size=1349654 2024-03-04T17:50:31.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.453+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~CG1DI53E0YML.0 size=1048576 2024-03-04T17:50:31.455 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.457+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~CG1DI53E0YML.1 size=785829 2024-03-04T17:50:31.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.461+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=CG1DI53E0YML tag=1LTWPF949EM9 type=write size=2882981 2024-03-04T17:50:31.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.469+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~PO7J1E5RG1BB.0 size=1048576 2024-03-04T17:50:31.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.473+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~PO7J1E5RG1BB.1 size=1048576 2024-03-04T17:50:31.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.477+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~PO7J1E5RG1BB.2 size=656225 2024-03-04T17:50:31.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.477+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=PO7J1E5RG1BB tag=V7ZCQCMSEZ7R type=write size=3801953 2024-03-04T17:50:31.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.481+0000 7fda08af98c0 1 < canceled operation key=C1GK00FM2ZYA tag=M2K3Q01D1MNB type=del size=3820943 2024-03-04T17:50:31.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.485+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=TT9FAEP577W6 type=del size=3348270 2024-03-04T17:50:31.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.489+0000 7fda08af98c0 1 < completed write operation key=8YL7C9PI58TK tag=0W1O049UAPPS type=write size=595775 2024-03-04T17:50:31.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.493+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=8K6S7VSBSIOM type=del size=693507 2024-03-04T17:50:31.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.497+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=MNBXGOXC8ZYM type=del 2024-03-04T17:50:31.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.501+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=4FQV7Y8RW4LZ type=del size=2043268 2024-03-04T17:50:31.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.505+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~X3HZW5EDDNGK.0 size=1048576 2024-03-04T17:50:31.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.505+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~X3HZW5EDDNGK.1 size=840512 2024-03-04T17:50:31.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.513+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=X3HZW5EDDNGK tag=TTUW7ZP0I0OW type=write size=2937664 2024-03-04T17:50:31.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.521+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~JM5L2L5P60Z6.0 size=1048576 2024-03-04T17:50:31.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.525+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~JM5L2L5P60Z6.1 size=1048576 2024-03-04T17:50:31.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.525+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~JM5L2L5P60Z6.2 size=931976 2024-03-04T17:50:31.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.525+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=JM5L2L5P60Z6 tag=QENX39ZPSDNK type=write size=4077704 2024-03-04T17:50:31.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.529+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~D3GVLUUZJ4ZX.0 size=1048576 2024-03-04T17:50:31.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.533+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~D3GVLUUZJ4ZX.1 size=1048576 2024-03-04T17:50:31.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.537+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~D3GVLUUZJ4ZX.2 size=569592 2024-03-04T17:50:31.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.541+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=D3GVLUUZJ4ZX tag=MBCOU0RX8K76 type=write size=3715320 2024-03-04T17:50:31.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.545+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=6MLMJW9CZ5AI type=write size=562204 2024-03-04T17:50:31.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.549+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~YETO0LGDB1LE.0 size=1048576 2024-03-04T17:50:31.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.553+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~YETO0LGDB1LE.1 size=1048576 2024-03-04T17:50:31.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.553+0000 7fda08af98c0 1 < canceled part key=_multipart_IJSYAVR5SWPS.2~YETO0LGDB1LE.2 size=202631 2024-03-04T17:50:31.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.557+0000 7fda08af98c0 1 < canceled multipart upload key=IJSYAVR5SWPS upload=YETO0LGDB1LE tag=160UCU3PS2UM type=write size=3348359 2024-03-04T17:50:31.563 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.565+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~L2U56NZGJPYH.0 size=1048576 2024-03-04T17:50:31.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.569+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~L2U56NZGJPYH.1 size=1048576 2024-03-04T17:50:31.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.569+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~L2U56NZGJPYH.2 size=668191 2024-03-04T17:50:31.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.569+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=L2U56NZGJPYH tag=6M55C0K2OIMJ type=write size=3813919 2024-03-04T17:50:31.573 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.573+0000 7fda08af98c0 1 < completed write operation key=3UWSL7NJLACY tag=4VKNQYZYBNGO type=write size=702350 2024-03-04T17:50:31.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.581+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~GOQ6Z6R2NWKF.0 size=1048576 2024-03-04T17:50:31.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.585+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~GOQ6Z6R2NWKF.1 size=1048576 2024-03-04T17:50:31.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.585+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~GOQ6Z6R2NWKF.2 size=359029 2024-03-04T17:50:31.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.585+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=GOQ6Z6R2NWKF tag=W3193E2I9ZJM type=write size=3504757 2024-03-04T17:50:31.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.589+0000 7fda08af98c0 1 < completed write operation key=C4B45GL40IMZ tag=56ZNN0JTNQDK type=write size=809158 2024-03-04T17:50:31.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.597+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~VVN9VSSYVKSR.0 size=1048576 2024-03-04T17:50:31.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.601+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~VVN9VSSYVKSR.1 size=1048576 2024-03-04T17:50:31.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.601+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~VVN9VSSYVKSR.2 size=585184 2024-03-04T17:50:31.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.601+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=VVN9VSSYVKSR tag=SLQ0XV9B6IZV type=write size=3730912 2024-03-04T17:50:31.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.605+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=LFJHA91UJ11N type=del 2024-03-04T17:50:31.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.609+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=41AWO1DZISTT type=del size=1487846 2024-03-04T17:50:31.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.613+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=4V04WGU7IDUN type=del 2024-03-04T17:50:31.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.617+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=DCV27ZM65405 type=del size=3218815 2024-03-04T17:50:31.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.621+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=5BO5Z3YWABQE type=del 2024-03-04T17:50:31.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.625+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=PR9MWKKR8GHI type=del size=440183 2024-03-04T17:50:31.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.629+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=03B2HEB0VXAA type=del 2024-03-04T17:50:31.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.633+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=SJWECCX80MI5 type=del size=4188029 2024-03-04T17:50:31.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.637+0000 7fda08af98c0 1 < canceled operation key=V1LDB2HXZ7EF tag=RHJFJYCW5K2C type=write size=1004884 2024-03-04T17:50:31.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.645+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~GBBPXVCFYD0R.0 size=1048576 2024-03-04T17:50:31.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.649+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~GBBPXVCFYD0R.1 size=166181 2024-03-04T17:50:31.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.649+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=GBBPXVCFYD0R tag=ZR60GZLK4ENS type=write size=2263333 2024-03-04T17:50:31.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.653+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~AS4ISH14RBUU.0 size=1048576 2024-03-04T17:50:31.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.653+0000 7fda08af98c0 1 < canceled part key=_multipart_3UWSL7NJLACY.2~AS4ISH14RBUU.1 size=1048576 2024-03-04T17:50:31.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.653+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~AS4ISH14RBUU.2 size=778357 2024-03-04T17:50:31.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.661+0000 7fda08af98c0 1 < canceled multipart upload key=3UWSL7NJLACY upload=AS4ISH14RBUU tag=27FZ985T5HBA type=write size=3924085 2024-03-04T17:50:31.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.669+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~ULR9KUHTANEL.0 size=1048576 2024-03-04T17:50:31.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.669+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~ULR9KUHTANEL.1 size=1048576 2024-03-04T17:50:31.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.673+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~ULR9KUHTANEL.2 size=318047 2024-03-04T17:50:31.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.673+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=ULR9KUHTANEL tag=2AV54S3BJWHK type=write size=3463775 2024-03-04T17:50:31.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.677+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=0XOAOAF9RVUL type=del 2024-03-04T17:50:31.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.681+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=KI7GCFFZ1P1X type=del size=309335 2024-03-04T17:50:31.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.685+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~PO7J1E5RG1BB.0 size=1048576 2024-03-04T17:50:31.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.689+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~PO7J1E5RG1BB.1 size=1048576 2024-03-04T17:50:31.689 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.689+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~PO7J1E5RG1BB.2 size=656225 2024-03-04T17:50:31.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.697+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=PO7J1E5RG1BB tag=V7ZCQCMSEZ7R type=write size=3801953 2024-03-04T17:50:31.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.705+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~SC0N246I8X2D.0 size=812039 2024-03-04T17:50:31.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.705+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=SC0N246I8X2D tag=FX82KITKBU63 type=write size=1860615 2024-03-04T17:50:31.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.713+0000 7fda08af98c0 1 < suggested update operation key=L0FVKWD7EQRW tag=TT9FAEP577W6 type=del size=3348270 2024-03-04T17:50:31.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.717+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=AAF4R58G6OF3 type=write size=137505 2024-03-04T17:50:31.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.721+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=8K6S7VSBSIOM type=del 2024-03-04T17:50:31.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.725+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=9AT6SX5JJRLP type=write size=727574 2024-03-04T17:50:31.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.729+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=4FQV7Y8RW4LZ type=del 2024-03-04T17:50:31.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.733+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=QPCRICDBED2L type=del size=3728286 2024-03-04T17:50:31.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.737+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~JM5L2L5P60Z6.0 size=1048576 2024-03-04T17:50:31.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.741+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~JM5L2L5P60Z6.1 size=1048576 2024-03-04T17:50:31.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.741+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~JM5L2L5P60Z6.2 size=931976 2024-03-04T17:50:31.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.749+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=JM5L2L5P60Z6 tag=QENX39ZPSDNK type=write size=4077704 2024-03-04T17:50:31.755 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.757+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~1CRHQVARQ1I9.0 size=486337 2024-03-04T17:50:31.755 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.757+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=1CRHQVARQ1I9 tag=BUGTVW1GA2WV type=write size=1534913 2024-03-04T17:50:31.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.761+0000 7fda08af98c0 1 < completed write operation key=AO3QS6CM7I5D tag=6MLMJW9CZ5AI type=write size=562204 2024-03-04T17:50:31.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.769+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~V06UHWWAELGT.0 size=1048576 2024-03-04T17:50:31.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.769+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~V06UHWWAELGT.1 size=938491 2024-03-04T17:50:31.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.769+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=V06UHWWAELGT tag=K2C105EI3W6S type=write size=3035643 2024-03-04T17:50:31.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.773+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~L2U56NZGJPYH.0 size=1048576 2024-03-04T17:50:31.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.777+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~L2U56NZGJPYH.1 size=1048576 2024-03-04T17:50:31.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.781+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~L2U56NZGJPYH.2 size=668191 2024-03-04T17:50:31.786 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.789+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=L2U56NZGJPYH tag=6M55C0K2OIMJ type=write size=3813919 2024-03-04T17:50:31.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.793+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=W71FTPDFCTYD type=del size=1414301 2024-03-04T17:50:31.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.797+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~GOQ6Z6R2NWKF.0 size=1048576 2024-03-04T17:50:31.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.801+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~GOQ6Z6R2NWKF.1 size=1048576 2024-03-04T17:50:31.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.805+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~GOQ6Z6R2NWKF.2 size=359029 2024-03-04T17:50:31.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.809+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=GOQ6Z6R2NWKF tag=W3193E2I9ZJM type=write size=3504757 2024-03-04T17:50:31.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.813+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=66QKHZ2P744L type=del size=3415287 2024-03-04T17:50:31.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.817+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~VVN9VSSYVKSR.0 size=1048576 2024-03-04T17:50:31.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.821+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~VVN9VSSYVKSR.1 size=1048576 2024-03-04T17:50:31.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.825+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~VVN9VSSYVKSR.2 size=585184 2024-03-04T17:50:31.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.829+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=VVN9VSSYVKSR tag=SLQ0XV9B6IZV type=write size=3730912 2024-03-04T17:50:31.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.833+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=OO5JB8R8NNDU type=write size=481219 2024-03-04T17:50:31.837 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.837+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=41AWO1DZISTT type=del 2024-03-04T17:50:31.844 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.845+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~1WC9EI28V0AK.0 size=1031441 2024-03-04T17:50:31.844 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.845+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=1WC9EI28V0AK tag=R12YH68HIA1Z type=write size=2080017 2024-03-04T17:50:31.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.849+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=DCV27ZM65405 type=del 2024-03-04T17:50:31.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.857+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~2LW3HWSNRNOZ.0 size=693726 2024-03-04T17:50:31.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.857+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=2LW3HWSNRNOZ tag=V54E2MOTQZ00 type=write size=1742302 2024-03-04T17:50:31.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.861+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=PR9MWKKR8GHI type=del 2024-03-04T17:50:31.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.865+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~SI4I9Y2F8YT7.0 size=359339 2024-03-04T17:50:31.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.865+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=SI4I9Y2F8YT7 tag=J4OHH6C8SLRL type=write size=1407915 2024-03-04T17:50:31.869 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.869+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=SJWECCX80MI5 type=del 2024-03-04T17:50:31.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.873+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=0BBRAV3QJLGZ type=del size=3447988 2024-03-04T17:50:31.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.877+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~GBBPXVCFYD0R.0 size=1048576 2024-03-04T17:50:31.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.881+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~GBBPXVCFYD0R.1 size=166181 2024-03-04T17:50:31.886 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.889+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=GBBPXVCFYD0R tag=ZR60GZLK4ENS type=write size=2263333 2024-03-04T17:50:31.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.893+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=F758DOJNB94L type=del size=3151242 2024-03-04T17:50:31.894 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.897+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~ULR9KUHTANEL.0 size=1048576 2024-03-04T17:50:31.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.901+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~ULR9KUHTANEL.1 size=1048576 2024-03-04T17:50:31.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.901+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~ULR9KUHTANEL.2 size=318047 2024-03-04T17:50:31.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.909+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=ULR9KUHTANEL tag=2AV54S3BJWHK type=write size=3463775 2024-03-04T17:50:31.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.913+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=KDSZSU8RNTM4 type=del size=2129991 2024-03-04T17:50:31.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.917+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=KI7GCFFZ1P1X type=del 2024-03-04T17:50:31.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.925+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~1SD3K7FSZ4ZV.0 size=1048576 2024-03-04T17:50:31.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.929+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~1SD3K7FSZ4ZV.1 size=1048576 2024-03-04T17:50:31.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.929+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~1SD3K7FSZ4ZV.2 size=35640 2024-03-04T17:50:31.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.929+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=1SD3K7FSZ4ZV tag=1PE8QIF4PVN7 type=write size=3181368 2024-03-04T17:50:31.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.933+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~SC0N246I8X2D.0 size=812039 2024-03-04T17:50:31.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.941+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=SC0N246I8X2D tag=FX82KITKBU63 type=write size=1860615 2024-03-04T17:50:31.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.949+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~TLQ7GIABP4QL.0 size=603618 2024-03-04T17:50:31.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.949+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=TLQ7GIABP4QL tag=54KCQBRWWRUF type=write size=1652194 2024-03-04T17:50:31.950 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.953+0000 7fda08af98c0 1 < completed write operation key=VJVORUG95FLC tag=AAF4R58G6OF3 type=write size=137505 2024-03-04T17:50:31.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.957+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~ZN1C3BFS2RI7.0 size=1048576 2024-03-04T17:50:31.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.961+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~ZN1C3BFS2RI7.1 size=1048576 2024-03-04T17:50:31.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.965+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~ZN1C3BFS2RI7.2 size=937648 2024-03-04T17:50:31.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.965+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=ZN1C3BFS2RI7 tag=JI7XSI1QOR3V type=write size=4083376 2024-03-04T17:50:31.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.969+0000 7fda08af98c0 1 < completed write operation key=TOFWDFH0HL3Q tag=9AT6SX5JJRLP type=write size=727574 2024-03-04T17:50:31.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.977+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~VJ7UV5V4C1CO.0 size=1048576 2024-03-04T17:50:31.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.977+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~VJ7UV5V4C1CO.1 size=1048576 2024-03-04T17:50:31.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.981+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~VJ7UV5V4C1CO.2 size=596509 2024-03-04T17:50:31.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.981+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=VJ7UV5V4C1CO tag=RNBX499BLRTZ type=write size=3742237 2024-03-04T17:50:31.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.989+0000 7fda08af98c0 1 < suggested remove operation key=RXVSQVVNXVF0 tag=QPCRICDBED2L type=del 2024-03-04T17:50:31.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.997+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~20TPFFY90WPZ.0 size=148550 2024-03-04T17:50:31.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:31.997+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=20TPFFY90WPZ tag=V7TPLPPII4U4 type=write size=1197126 2024-03-04T17:50:31.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.001+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~1CRHQVARQ1I9.0 size=486337 2024-03-04T17:50:32.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.005+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=1CRHQVARQ1I9 tag=BUGTVW1GA2WV type=write size=1534913 2024-03-04T17:50:32.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.009+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=MF1HSG1J0UAI type=write size=550994 2024-03-04T17:50:32.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.013+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~V06UHWWAELGT.0 size=1048576 2024-03-04T17:50:32.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.017+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~V06UHWWAELGT.1 size=938491 2024-03-04T17:50:32.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.025+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=V06UHWWAELGT tag=K2C105EI3W6S type=write size=3035643 2024-03-04T17:50:32.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.029+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=XAHSIOL6R3L3 type=del size=1968277 2024-03-04T17:50:32.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.033+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=W71FTPDFCTYD type=del 2024-03-04T17:50:32.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.037+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=Q864P08X3SN4 type=del size=4061000 2024-03-04T17:50:32.039 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.041+0000 7fda08af98c0 1 < canceled operation key=HCA7QLGM2O4O tag=66QKHZ2P744L type=del size=3415287 2024-03-04T17:50:32.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.045+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=NNRKVUT9QS8N type=del size=3362583 2024-03-04T17:50:32.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.049+0000 7fda08af98c0 1 < completed write operation key=09KBCA2XURTF tag=OO5JB8R8NNDU type=write size=481219 2024-03-04T17:50:32.051 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.053+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=FEYGCBE85KI9 type=del size=1118102 2024-03-04T17:50:32.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.057+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~1WC9EI28V0AK.0 size=1031441 2024-03-04T17:50:32.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.061+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=1WC9EI28V0AK tag=R12YH68HIA1Z type=write size=2080017 2024-03-04T17:50:32.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.065+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=NXKKR21QQUWQ type=del size=1399911 2024-03-04T17:50:32.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.069+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~2LW3HWSNRNOZ.0 size=693726 2024-03-04T17:50:32.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.077+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=2LW3HWSNRNOZ tag=V54E2MOTQZ00 type=write size=1742302 2024-03-04T17:50:32.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.081+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=9OYYG4S3XZNN type=del size=2780293 2024-03-04T17:50:32.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.085+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~SI4I9Y2F8YT7.0 size=359339 2024-03-04T17:50:32.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.089+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=SI4I9Y2F8YT7 tag=J4OHH6C8SLRL type=write size=1407915 2024-03-04T17:50:32.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.097+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~ZNN10P2F8OVU.0 size=1048576 2024-03-04T17:50:32.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.101+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~ZNN10P2F8OVU.1 size=846465 2024-03-04T17:50:32.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.101+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=ZNN10P2F8OVU tag=MENO95IZ5VKF type=write size=2943617 2024-03-04T17:50:32.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.105+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=0BBRAV3QJLGZ type=del 2024-03-04T17:50:32.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.113+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~2BCFVTD76ZQ0.0 size=1048576 2024-03-04T17:50:32.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.117+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~2BCFVTD76ZQ0.1 size=1048576 2024-03-04T17:50:32.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.121+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~2BCFVTD76ZQ0.2 size=1022094 2024-03-04T17:50:32.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.121+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=2BCFVTD76ZQ0 tag=9JMNHJ3ZE354 type=write size=4167822 2024-03-04T17:50:32.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.125+0000 7fda08af98c0 1 < suggested remove operation key=X5MEJ7G8X9HE tag=F758DOJNB94L type=del 2024-03-04T17:50:32.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.133+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~1E7QOQ15309B.0 size=939516 2024-03-04T17:50:32.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.133+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=1E7QOQ15309B tag=PT5S5IUWD0WS type=write size=1988092 2024-03-04T17:50:32.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.137+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=KDSZSU8RNTM4 type=del 2024-03-04T17:50:32.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.141+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=8YSMR0FF6MFM type=write size=811832 2024-03-04T17:50:32.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.145+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~1SD3K7FSZ4ZV.0 size=1048576 2024-03-04T17:50:32.148 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.149+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~1SD3K7FSZ4ZV.1 size=1048576 2024-03-04T17:50:32.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.157+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~1SD3K7FSZ4ZV.2 size=35640 2024-03-04T17:50:32.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.161+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=1SD3K7FSZ4ZV tag=1PE8QIF4PVN7 type=write size=3181368 2024-03-04T17:50:32.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.165+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=UFP2MDUXDA1H type=del size=3517991 2024-03-04T17:50:32.168 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.169+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~TLQ7GIABP4QL.0 size=603618 2024-03-04T17:50:32.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.177+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=TLQ7GIABP4QL tag=54KCQBRWWRUF type=write size=1652194 2024-03-04T17:50:32.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.181+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~5TINWHCKC55B.0 size=1048576 2024-03-04T17:50:32.183 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.185+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~5TINWHCKC55B.1 size=1048576 2024-03-04T17:50:32.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.185+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~5TINWHCKC55B.2 size=864873 2024-03-04T17:50:32.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.185+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=5TINWHCKC55B tag=YG32Z97XF7DY type=write size=4010601 2024-03-04T17:50:32.189 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.189+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~ZN1C3BFS2RI7.0 size=1048576 2024-03-04T17:50:32.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.193+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~ZN1C3BFS2RI7.1 size=1048576 2024-03-04T17:50:32.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.197+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~ZN1C3BFS2RI7.2 size=937648 2024-03-04T17:50:32.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.201+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=ZN1C3BFS2RI7 tag=JI7XSI1QOR3V type=write size=4083376 2024-03-04T17:50:32.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.209+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~YV6YXZ73CKFB.0 size=1048576 2024-03-04T17:50:32.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.213+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~YV6YXZ73CKFB.1 size=1048576 2024-03-04T17:50:32.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.213+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~YV6YXZ73CKFB.2 size=571080 2024-03-04T17:50:32.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.213+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=YV6YXZ73CKFB tag=I8Z325PUZCV2 type=write size=3716808 2024-03-04T17:50:32.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.217+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~VJ7UV5V4C1CO.0 size=1048576 2024-03-04T17:50:32.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.217+0000 7fda08af98c0 1 < canceled part key=_multipart_BYQFZSJ3WH07.2~VJ7UV5V4C1CO.1 size=1048576 2024-03-04T17:50:32.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.221+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~VJ7UV5V4C1CO.2 size=596509 2024-03-04T17:50:32.225 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.225+0000 7fda08af98c0 1 < canceled multipart upload key=BYQFZSJ3WH07 upload=VJ7UV5V4C1CO tag=RNBX499BLRTZ type=write size=3742237 2024-03-04T17:50:32.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.233+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~SODD4C0JMH0O.0 size=192854 2024-03-04T17:50:32.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.233+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=SODD4C0JMH0O tag=I73J1643OYVC type=write size=1241430 2024-03-04T17:50:32.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.237+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~20TPFFY90WPZ.0 size=148550 2024-03-04T17:50:32.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.245+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=20TPFFY90WPZ tag=V7TPLPPII4U4 type=write size=1197126 2024-03-04T17:50:32.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.249+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=X3I2341GIUMX type=del size=231819 2024-03-04T17:50:32.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.257+0000 7fda08af98c0 1 < suggested update operation key=Z3J7W9V8HQKH tag=MF1HSG1J0UAI type=write size=550994 2024-03-04T17:50:32.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.261+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=YCNKTZDC7KYM type=del size=2026689 2024-03-04T17:50:32.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.269+0000 7fda08af98c0 1 < suggested remove operation key=09KBCA2XURTF tag=XAHSIOL6R3L3 type=del 2024-03-04T17:50:32.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.273+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=EPBMXCADNB2E type=write size=924114 2024-03-04T17:50:32.277 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.277+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=Q864P08X3SN4 type=del 2024-03-04T17:50:32.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.289+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~KCZJO2GSBBE4.0 size=701978 2024-03-04T17:50:32.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.289+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=KCZJO2GSBBE4 tag=43VPH2MAPIVQ type=write size=1750554 2024-03-04T17:50:32.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.289+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=NNRKVUT9QS8N type=del 2024-03-04T17:50:32.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.293+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=2G8G0VBR2T9O type=del size=1928717 2024-03-04T17:50:32.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.301+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=FEYGCBE85KI9 type=del 2024-03-04T17:50:32.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.305+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=9PM1ONJW2YE2 type=write size=825177 2024-03-04T17:50:32.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.313+0000 7fda08af98c0 1 < suggested update operation key=750NSVFBS03A tag=NXKKR21QQUWQ type=del size=1399911 2024-03-04T17:50:32.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.317+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=A9DV7CU36IVA type=del size=1640277 2024-03-04T17:50:32.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.321+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=9OYYG4S3XZNN type=del 2024-03-04T17:50:32.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.325+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=XNPTIXJ302GI type=del size=606976 2024-03-04T17:50:32.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.329+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~ZNN10P2F8OVU.0 size=1048576 2024-03-04T17:50:32.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.333+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~ZNN10P2F8OVU.1 size=846465 2024-03-04T17:50:32.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.337+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=ZNN10P2F8OVU tag=MENO95IZ5VKF type=write size=2943617 2024-03-04T17:50:32.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.341+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=HEEQVT41GWSX type=write size=581612 2024-03-04T17:50:32.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.345+0000 7fda08af98c0 1 < canceled part key=_multipart_750NSVFBS03A.2~2BCFVTD76ZQ0.0 size=1048576 2024-03-04T17:50:32.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.349+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~2BCFVTD76ZQ0.1 size=1048576 2024-03-04T17:50:32.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.349+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~2BCFVTD76ZQ0.2 size=1022094 2024-03-04T17:50:32.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.357+0000 7fda08af98c0 1 < canceled multipart upload key=750NSVFBS03A upload=2BCFVTD76ZQ0 tag=9JMNHJ3ZE354 type=write size=4167822 2024-03-04T17:50:32.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.361+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=TGMFQ1YWN57U type=write size=678163 2024-03-04T17:50:32.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.365+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~1E7QOQ15309B.0 size=939516 2024-03-04T17:50:32.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.369+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=1E7QOQ15309B tag=PT5S5IUWD0WS type=write size=1988092 2024-03-04T17:50:32.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.373+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=DP6B1F2DDYHT type=del size=1324063 2024-03-04T17:50:32.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.381+0000 7fda08af98c0 1 < suggested remove operation key=2PA8JDZUOLO5 tag=8YSMR0FF6MFM type=write 2024-03-04T17:50:32.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.389+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~YLIIXQVXDZV5.0 size=1048576 2024-03-04T17:50:32.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.393+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~YLIIXQVXDZV5.1 size=701779 2024-03-04T17:50:32.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.393+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=YLIIXQVXDZV5 tag=9C7F4WO6ME4K type=write size=2798931 2024-03-04T17:50:32.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.393+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=UFP2MDUXDA1H type=del 2024-03-04T17:50:32.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.401+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~34VEMWULZ85K.0 size=1048576 2024-03-04T17:50:32.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.405+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~34VEMWULZ85K.1 size=1048576 2024-03-04T17:50:32.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.405+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~34VEMWULZ85K.2 size=964258 2024-03-04T17:50:32.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.405+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=34VEMWULZ85K tag=PCVYSQ4OK8RV type=write size=4109986 2024-03-04T17:50:32.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.409+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~5TINWHCKC55B.0 size=1048576 2024-03-04T17:50:32.412 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.413+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~5TINWHCKC55B.1 size=1048576 2024-03-04T17:50:32.414 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.417+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~5TINWHCKC55B.2 size=864873 2024-03-04T17:50:32.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.421+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=5TINWHCKC55B tag=YG32Z97XF7DY type=write size=4010601 2024-03-04T17:50:32.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.425+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=RKEPF101X6P8 type=del size=3891856 2024-03-04T17:50:32.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.429+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~YV6YXZ73CKFB.0 size=1048576 2024-03-04T17:50:32.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.433+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~YV6YXZ73CKFB.1 size=1048576 2024-03-04T17:50:32.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.437+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~YV6YXZ73CKFB.2 size=571080 2024-03-04T17:50:32.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.441+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=YV6YXZ73CKFB tag=I8Z325PUZCV2 type=write size=3716808 2024-03-04T17:50:32.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.445+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=22D5J8WS1HYM type=del size=1051908 2024-03-04T17:50:32.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.449+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~SODD4C0JMH0O.0 size=192854 2024-03-04T17:50:32.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.453+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=SODD4C0JMH0O tag=I73J1643OYVC type=write size=1241430 2024-03-04T17:50:32.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.461+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=1W5DW2QUN34W type=write size=711434 2024-03-04T17:50:32.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.465+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=X3I2341GIUMX type=del 2024-03-04T17:50:32.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.473+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~26TQV2N5DB8C.0 size=1048576 2024-03-04T17:50:32.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.477+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~26TQV2N5DB8C.1 size=1048576 2024-03-04T17:50:32.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.477+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~26TQV2N5DB8C.2 size=797864 2024-03-04T17:50:32.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.477+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=26TQV2N5DB8C tag=HXDRI4E1ETIF type=write size=3943592 2024-03-04T17:50:32.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.481+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=YCNKTZDC7KYM type=del 2024-03-04T17:50:32.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.489+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~P8PAUU8OY92B.0 size=1048576 2024-03-04T17:50:32.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.493+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~P8PAUU8OY92B.1 size=1048576 2024-03-04T17:50:32.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.497+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~P8PAUU8OY92B.2 size=129890 2024-03-04T17:50:32.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.497+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=P8PAUU8OY92B tag=10U0HYNJSBQ6 type=write size=3275618 2024-03-04T17:50:32.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.501+0000 7fda08af98c0 1 < canceled operation key=DJCCQ6S0E7UG tag=EPBMXCADNB2E type=write size=924114 2024-03-04T17:50:32.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.505+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=C2ZUZHLYLMZE type=del size=2879403 2024-03-04T17:50:32.509 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.509+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~KCZJO2GSBBE4.0 size=701978 2024-03-04T17:50:32.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.517+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=KCZJO2GSBBE4 tag=43VPH2MAPIVQ type=write size=1750554 2024-03-04T17:50:32.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.521+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=SBE2B5XPPJAY type=del size=1833812 2024-03-04T17:50:32.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.525+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=2G8G0VBR2T9O type=del 2024-03-04T17:50:32.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.529+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=YDNTITU0LDJ0 type=del size=1727211 2024-03-04T17:50:32.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.529+0000 7fda08af98c0 1 < completed write operation key=3UWSL7NJLACY tag=9PM1ONJW2YE2 type=write size=825177 2024-03-04T17:50:32.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.533+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=BCYO8VS42YML type=del size=3918948 2024-03-04T17:50:32.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.537+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=A9DV7CU36IVA type=del 2024-03-04T17:50:32.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.541+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=7EYWTYCLVMEF type=write size=850358 2024-03-04T17:50:32.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.545+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=XNPTIXJ302GI type=del 2024-03-04T17:50:32.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.549+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=DEE3HG3PKGPM type=del size=2618287 2024-03-04T17:50:32.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.557+0000 7fda08af98c0 1 < suggested remove operation key=C4B45GL40IMZ tag=HEEQVT41GWSX type=write 2024-03-04T17:50:32.559 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.561+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=WUHXUHDW6YN0 type=del size=671886 2024-03-04T17:50:32.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.565+0000 7fda08af98c0 1 < canceled operation key=L0FVKWD7EQRW tag=TGMFQ1YWN57U type=write size=678163 2024-03-04T17:50:32.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.569+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=J1L8B90SY9MW type=write size=352784 2024-03-04T17:50:32.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.573+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=DP6B1F2DDYHT type=del 2024-03-04T17:50:32.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.577+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=CNDCC69P9UEX type=del size=3590691 2024-03-04T17:50:32.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.581+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~YLIIXQVXDZV5.0 size=1048576 2024-03-04T17:50:32.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.585+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~YLIIXQVXDZV5.1 size=701779 2024-03-04T17:50:32.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.589+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=YLIIXQVXDZV5 tag=9C7F4WO6ME4K type=write size=2798931 2024-03-04T17:50:32.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.593+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=ZW2UTL7ALX84 type=del size=2293759 2024-03-04T17:50:32.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.597+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~34VEMWULZ85K.0 size=1048576 2024-03-04T17:50:32.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.601+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~34VEMWULZ85K.1 size=1048576 2024-03-04T17:50:32.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.601+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~34VEMWULZ85K.2 size=964258 2024-03-04T17:50:32.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.609+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=34VEMWULZ85K tag=PCVYSQ4OK8RV type=write size=4109986 2024-03-04T17:50:32.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.613+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=2R9D6VQZBBEL type=del size=3077949 2024-03-04T17:50:32.615 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.617+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=RKEPF101X6P8 type=del 2024-03-04T17:50:32.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.621+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~KOTXP5U04D2X.0 size=973522 2024-03-04T17:50:32.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.621+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=KOTXP5U04D2X tag=40Q7KTS0NSBC type=write size=2022098 2024-03-04T17:50:32.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.625+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=22D5J8WS1HYM type=del 2024-03-04T17:50:32.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.629+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=A00Y8QAT93YA type=del size=988537 2024-03-04T17:50:32.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.637+0000 7fda08af98c0 1 < completed write operation key=G0BHMSCO0XLF tag=1W5DW2QUN34W type=write size=711434 2024-03-04T17:50:32.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.641+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=WGM1E8IBOU9H type=del size=2312731 2024-03-04T17:50:32.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.645+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~26TQV2N5DB8C.0 size=1048576 2024-03-04T17:50:32.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.645+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~26TQV2N5DB8C.1 size=1048576 2024-03-04T17:50:32.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.649+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~26TQV2N5DB8C.2 size=797864 2024-03-04T17:50:32.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.657+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=26TQV2N5DB8C tag=HXDRI4E1ETIF type=write size=3943592 2024-03-04T17:50:32.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.661+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=7KM2LSOUYFYW type=write size=481059 2024-03-04T17:50:32.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.661+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~P8PAUU8OY92B.0 size=1048576 2024-03-04T17:50:32.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.665+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~P8PAUU8OY92B.1 size=1048576 2024-03-04T17:50:32.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.665+0000 7fda08af98c0 1 < canceled part key=_multipart_V1LDB2HXZ7EF.2~P8PAUU8OY92B.2 size=129890 2024-03-04T17:50:32.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.673+0000 7fda08af98c0 1 < canceled multipart upload key=V1LDB2HXZ7EF upload=P8PAUU8OY92B tag=10U0HYNJSBQ6 type=write size=3275618 2024-03-04T17:50:32.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.677+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=GOR1Q8HUOF24 type=del size=1669986 2024-03-04T17:50:32.677 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.677+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=C2ZUZHLYLMZE type=del 2024-03-04T17:50:32.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.685+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~YEE6BBM7UE5Z.0 size=1048576 2024-03-04T17:50:32.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.689+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~YEE6BBM7UE5Z.1 size=1048576 2024-03-04T17:50:32.689 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.693+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~YEE6BBM7UE5Z.2 size=240235 2024-03-04T17:50:32.690 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.693+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=YEE6BBM7UE5Z tag=YV586XR6IZK0 type=write size=3385963 2024-03-04T17:50:32.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.697+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=SBE2B5XPPJAY type=del 2024-03-04T17:50:32.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.701+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~U1WNGBXWXPJ9.0 size=1048576 2024-03-04T17:50:32.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.705+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~U1WNGBXWXPJ9.1 size=473532 2024-03-04T17:50:32.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.705+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=U1WNGBXWXPJ9 tag=EKCOPUHGMAGP type=write size=2570684 2024-03-04T17:50:32.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.709+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=YDNTITU0LDJ0 type=del 2024-03-04T17:50:32.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.713+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=7UW0YP9ZXXCP type=del size=72985 2024-03-04T17:50:32.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.717+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=BCYO8VS42YML type=del 2024-03-04T17:50:32.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.721+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=E2QSIGD7MVRF type=del size=1713426 2024-03-04T17:50:32.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.725+0000 7fda08af98c0 1 < completed write operation key=09KBCA2XURTF tag=7EYWTYCLVMEF type=write size=850358 2024-03-04T17:50:32.728 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.729+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~F445G6UFGPMR.0 size=1048576 2024-03-04T17:50:32.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.733+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~F445G6UFGPMR.1 size=766022 2024-03-04T17:50:32.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.733+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=F445G6UFGPMR tag=GXWWDU9LFF6Z type=write size=2863174 2024-03-04T17:50:32.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.737+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=DEE3HG3PKGPM type=del 2024-03-04T17:50:32.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.741+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=1CZE83LXA16M type=del size=3973651 2024-03-04T17:50:32.742 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.745+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=WUHXUHDW6YN0 type=del 2024-03-04T17:50:32.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.749+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=DXYBXFZTKXLJ type=del size=1526681 2024-03-04T17:50:32.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.753+0000 7fda08af98c0 1 < completed write operation key=8YL7C9PI58TK tag=J1L8B90SY9MW type=write size=352784 2024-03-04T17:50:32.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.753+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=ESYPLUJQPIRF type=del size=928402 2024-03-04T17:50:32.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.757+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=CNDCC69P9UEX type=del 2024-03-04T17:50:32.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.761+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=QPL0PAQC1UQB type=del size=2053554 2024-03-04T17:50:32.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.765+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=ZW2UTL7ALX84 type=del 2024-03-04T17:50:32.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.773+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=5NJFW0C60992 type=write size=742935 2024-03-04T17:50:32.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.777+0000 7fda08af98c0 1 < suggested update operation key=AO3QS6CM7I5D tag=2R9D6VQZBBEL type=del size=3077949 2024-03-04T17:50:32.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.785+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~2UTP1V474YLT.0 size=1048576 2024-03-04T17:50:32.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.789+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~2UTP1V474YLT.1 size=518620 2024-03-04T17:50:32.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.789+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=2UTP1V474YLT tag=HRJ5EHFS1EL2 type=write size=2615772 2024-03-04T17:50:32.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.793+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~KOTXP5U04D2X.0 size=973522 2024-03-04T17:50:32.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.801+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=KOTXP5U04D2X tag=40Q7KTS0NSBC type=write size=2022098 2024-03-04T17:50:32.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.805+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=6BZNNKKKLQK1 type=del size=4152028 2024-03-04T17:50:32.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.813+0000 7fda08af98c0 1 < suggested update operation key=Z2JQBN84QQIB tag=A00Y8QAT93YA type=del size=988537 2024-03-04T17:50:32.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.821+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~RS9V3FK5LKKE.0 size=1048576 2024-03-04T17:50:32.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.821+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~RS9V3FK5LKKE.1 size=258669 2024-03-04T17:50:32.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.821+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=RS9V3FK5LKKE tag=FMD9J9MCR89Z type=write size=2355821 2024-03-04T17:50:32.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.825+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=WGM1E8IBOU9H type=del 2024-03-04T17:50:32.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.829+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=VKPW0VJHYM9Z type=del size=2594767 2024-03-04T17:50:32.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.837+0000 7fda08af98c0 1 < suggested update operation key=1OZKPHMPV3OQ tag=7KM2LSOUYFYW type=write size=481059 2024-03-04T17:50:32.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.845+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~B8GE0YTGBUGS.0 size=1048576 2024-03-04T17:50:32.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.849+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~B8GE0YTGBUGS.1 size=149580 2024-03-04T17:50:32.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.849+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=B8GE0YTGBUGS tag=GDMTC4RQ8OTI type=write size=2246732 2024-03-04T17:50:32.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.857+0000 7fda08af98c0 1 < suggested remove operation key=VJVORUG95FLC tag=GOR1Q8HUOF24 type=del 2024-03-04T17:50:32.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.861+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=URCM2G5FKOCY type=write size=789085 2024-03-04T17:50:32.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.865+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~YEE6BBM7UE5Z.0 size=1048576 2024-03-04T17:50:32.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.869+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~YEE6BBM7UE5Z.1 size=1048576 2024-03-04T17:50:32.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.873+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~YEE6BBM7UE5Z.2 size=240235 2024-03-04T17:50:32.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.877+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=YEE6BBM7UE5Z tag=YV586XR6IZK0 type=write size=3385963 2024-03-04T17:50:32.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.881+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=MKLGFDYIZFUR type=del size=1185169 2024-03-04T17:50:32.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.885+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~U1WNGBXWXPJ9.0 size=1048576 2024-03-04T17:50:32.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.889+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~U1WNGBXWXPJ9.1 size=473532 2024-03-04T17:50:32.893 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.893+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=U1WNGBXWXPJ9 tag=EKCOPUHGMAGP type=write size=2570684 2024-03-04T17:50:32.900 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.901+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~OE3CY3RU9ADW.0 size=773006 2024-03-04T17:50:32.900 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.901+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=OE3CY3RU9ADW tag=YA7GTFF3IBMH type=write size=1821582 2024-03-04T17:50:32.903 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.905+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=7UW0YP9ZXXCP type=del size=72985 2024-03-04T17:50:32.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.909+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=7DELMHXBWKGJ type=write size=55670 2024-03-04T17:50:32.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.913+0000 7fda08af98c0 -1 < failed to complete operation key=AO3QS6CM7I5D tag=E2QSIGD7MVRF type=del size=1713426: (22) Invalid argument 2024-03-04T17:50:32.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.921+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~38ONSZNT2URI.0 size=38236 2024-03-04T17:50:32.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.921+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=38ONSZNT2URI tag=ALKRGJHQ27QJ type=write size=1086812 2024-03-04T17:50:32.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.925+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~F445G6UFGPMR.0 size=1048576 2024-03-04T17:50:32.925 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.925+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~F445G6UFGPMR.1 size=766022 2024-03-04T17:50:32.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.933+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=F445G6UFGPMR tag=GXWWDU9LFF6Z type=write size=2863174 2024-03-04T17:50:32.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.937+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=ND26YNX94YXT type=del size=325349 2024-03-04T17:50:32.944 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.945+0000 7fda08af98c0 1 < suggested update operation key=TOFWDFH0HL3Q tag=1CZE83LXA16M type=del size=3973651 2024-03-04T17:50:32.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.949+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=Z26TI3M33YG0 type=del size=407735 2024-03-04T17:50:32.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.953+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=DXYBXFZTKXLJ type=del 2024-03-04T17:50:32.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.961+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~RX1MD8D5HZOM.0 size=1048576 2024-03-04T17:50:32.964 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.961+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~RX1MD8D5HZOM.1 size=726677 2024-03-04T17:50:32.964 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.961+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=RX1MD8D5HZOM tag=P7HYFTOIU5X5 type=write size=2823829 2024-03-04T17:50:32.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.969+0000 7fda08af98c0 1 < suggested remove operation key=3UWSL7NJLACY tag=ESYPLUJQPIRF type=del 2024-03-04T17:50:32.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.977+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~HHO69VQFI43C.0 size=1048576 2024-03-04T17:50:32.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.977+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~HHO69VQFI43C.1 size=1048576 2024-03-04T17:50:32.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.981+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~HHO69VQFI43C.2 size=948884 2024-03-04T17:50:32.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.981+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=HHO69VQFI43C tag=KUWJSOIJVN7W type=write size=4094612 2024-03-04T17:50:32.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.985+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=QPL0PAQC1UQB type=del 2024-03-04T17:50:32.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.993+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~BVXQJYLITRMD.0 size=240984 2024-03-04T17:50:32.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.993+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=BVXQJYLITRMD tag=IR8Z7TYSSP94 type=write size=1289560 2024-03-04T17:50:32.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:32.997+0000 7fda08af98c0 1 < completed write operation key=M5BMX7B7RZYL tag=5NJFW0C60992 type=write size=742935 2024-03-04T17:50:33.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.001+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~M4UF2UE9DYV4.0 size=1048576 2024-03-04T17:50:33.003 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.005+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~M4UF2UE9DYV4.1 size=1048576 2024-03-04T17:50:33.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.009+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~M4UF2UE9DYV4.2 size=149781 2024-03-04T17:50:33.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.009+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=M4UF2UE9DYV4 tag=Z6BNXEI52GO9 type=write size=3295509 2024-03-04T17:50:33.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.013+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~2UTP1V474YLT.0 size=1048576 2024-03-04T17:50:33.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.013+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~2UTP1V474YLT.1 size=518620 2024-03-04T17:50:33.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.021+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=2UTP1V474YLT tag=HRJ5EHFS1EL2 type=write size=2615772 2024-03-04T17:50:33.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.025+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=MBWF6W9IWNZL type=write size=524864 2024-03-04T17:50:33.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.029+0000 7fda08af98c0 1 < canceled operation key=BYQFZSJ3WH07 tag=6BZNNKKKLQK1 type=del size=4152028 2024-03-04T17:50:33.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.033+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=HIVPGCLE7X96 type=del size=4112301 2024-03-04T17:50:33.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.037+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~RS9V3FK5LKKE.0 size=1048576 2024-03-04T17:50:33.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.041+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~RS9V3FK5LKKE.1 size=258669 2024-03-04T17:50:33.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.045+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=RS9V3FK5LKKE tag=FMD9J9MCR89Z type=write size=2355821 2024-03-04T17:50:33.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.049+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=UZ5SX4GZX0VX type=del size=2938669 2024-03-04T17:50:33.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.053+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=VKPW0VJHYM9Z type=del 2024-03-04T17:50:33.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.057+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=AFVAVE1GRHYT type=del size=3751502 2024-03-04T17:50:33.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.061+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~B8GE0YTGBUGS.0 size=1048576 2024-03-04T17:50:33.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.065+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~B8GE0YTGBUGS.1 size=149580 2024-03-04T17:50:33.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.069+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=B8GE0YTGBUGS tag=GDMTC4RQ8OTI type=write size=2246732 2024-03-04T17:50:33.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.077+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~T2U6JB7NM89U.0 size=383171 2024-03-04T17:50:33.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.077+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=T2U6JB7NM89U tag=6K7CP0V4G01J type=write size=1431747 2024-03-04T17:50:33.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.081+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=URCM2G5FKOCY type=write size=789085 2024-03-04T17:50:33.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.085+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~XYN8HHSJOQDK.0 size=1048576 2024-03-04T17:50:33.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.089+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~XYN8HHSJOQDK.1 size=1048576 2024-03-04T17:50:33.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.093+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~XYN8HHSJOQDK.2 size=172388 2024-03-04T17:50:33.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.093+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=XYN8HHSJOQDK tag=D5UMTY2E55SO type=write size=3318116 2024-03-04T17:50:33.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.097+0000 7fda08af98c0 1 < canceled operation key=C4B45GL40IMZ tag=MKLGFDYIZFUR type=del size=1185169 2024-03-04T17:50:33.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.101+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=8ONDD1NAC4ID type=del size=3654714 2024-03-04T17:50:33.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.105+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~OE3CY3RU9ADW.0 size=773006 2024-03-04T17:50:33.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.109+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=OE3CY3RU9ADW tag=YA7GTFF3IBMH type=write size=1821582 2024-03-04T17:50:33.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.117+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~PBOYIP2MU66X.0 size=761191 2024-03-04T17:50:33.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.117+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=PBOYIP2MU66X tag=P24Y5GAWVVAU type=write size=1809767 2024-03-04T17:50:33.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.121+0000 7fda08af98c0 -1 < failed to complete operation key=3UWSL7NJLACY tag=7DELMHXBWKGJ type=write size=55670: (22) Invalid argument 2024-03-04T17:50:33.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.125+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=J7G4Q2ZDQ1G2 type=del size=3351683 2024-03-04T17:50:33.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.129+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~38ONSZNT2URI.0 size=38236 2024-03-04T17:50:33.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.133+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=38ONSZNT2URI tag=ALKRGJHQ27QJ type=write size=1086812 2024-03-04T17:50:33.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.141+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~PBNJOZ1VE0S0.0 size=1048576 2024-03-04T17:50:33.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.145+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~PBNJOZ1VE0S0.1 size=596744 2024-03-04T17:50:33.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.145+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=PBNJOZ1VE0S0 tag=HQNKYWRCJEGT type=write size=2693896 2024-03-04T17:50:33.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.149+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=ND26YNX94YXT type=del 2024-03-04T17:50:33.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.153+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~C987YR3AX98W.0 size=1048576 2024-03-04T17:50:33.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.157+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~C987YR3AX98W.1 size=1048576 2024-03-04T17:50:33.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.161+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~C987YR3AX98W.2 size=283818 2024-03-04T17:50:33.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.161+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=C987YR3AX98W tag=JEOLLQU8EJ3H type=write size=3429546 2024-03-04T17:50:33.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.165+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=Z26TI3M33YG0 type=del 2024-03-04T17:50:33.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.169+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=QWLMRRHL6I3R type=write size=603818 2024-03-04T17:50:33.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.173+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~RX1MD8D5HZOM.0 size=1048576 2024-03-04T17:50:33.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.177+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~RX1MD8D5HZOM.1 size=726677 2024-03-04T17:50:33.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.181+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=RX1MD8D5HZOM tag=P7HYFTOIU5X5 type=write size=2823829 2024-03-04T17:50:33.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.185+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=62ZGDCQRTH9T type=del size=2353713 2024-03-04T17:50:33.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.189+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~HHO69VQFI43C.0 size=1048576 2024-03-04T17:50:33.190 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.193+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~HHO69VQFI43C.1 size=1048576 2024-03-04T17:50:33.190 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.193+0000 7fda08af98c0 1 < canceled part key=_multipart_M5BMX7B7RZYL.2~HHO69VQFI43C.2 size=948884 2024-03-04T17:50:33.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.197+0000 7fda08af98c0 1 < canceled multipart upload key=M5BMX7B7RZYL upload=HHO69VQFI43C tag=KUWJSOIJVN7W type=write size=4094612 2024-03-04T17:50:33.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.201+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=67OULN1NBVLF type=write size=720692 2024-03-04T17:50:33.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.209+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~BVXQJYLITRMD.0 size=240984 2024-03-04T17:50:33.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.213+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=BVXQJYLITRMD tag=IR8Z7TYSSP94 type=write size=1289560 2024-03-04T17:50:33.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.217+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=B390UTO38KP1 type=del size=3429914 2024-03-04T17:50:33.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.221+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~M4UF2UE9DYV4.0 size=1048576 2024-03-04T17:50:33.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.225+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~M4UF2UE9DYV4.1 size=1048576 2024-03-04T17:50:33.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.229+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~M4UF2UE9DYV4.2 size=149781 2024-03-04T17:50:33.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.233+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=M4UF2UE9DYV4 tag=Z6BNXEI52GO9 type=write size=3295509 2024-03-04T17:50:33.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.237+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=SRCEKD0S0FA5 type=del size=863582 2024-03-04T17:50:33.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.241+0000 7fda08af98c0 1 < completed write operation key=L0FVKWD7EQRW tag=MBWF6W9IWNZL type=write size=524864 2024-03-04T17:50:33.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.245+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=IART95URPGTP type=del size=1624839 2024-03-04T17:50:33.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.253+0000 7fda08af98c0 1 < suggested update operation key=XG2XYBCMUNY5 tag=HIVPGCLE7X96 type=del size=4112301 2024-03-04T17:50:33.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.257+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=4VP4BYTRY4WH type=del size=3848463 2024-03-04T17:50:33.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.261+0000 7fda08af98c0 1 < canceled operation key=IJSYAVR5SWPS tag=UZ5SX4GZX0VX type=del size=2938669 2024-03-04T17:50:33.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.265+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=DMCO7AA7KE2D type=del size=630182 2024-03-04T17:50:33.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.273+0000 7fda08af98c0 1 < suggested remove operation key=1OZKPHMPV3OQ tag=AFVAVE1GRHYT type=del 2024-03-04T17:50:33.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.277+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=4QBMC8TU4FB5 type=del size=1910996 2024-03-04T17:50:33.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.281+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~T2U6JB7NM89U.0 size=383171 2024-03-04T17:50:33.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.285+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=T2U6JB7NM89U tag=6K7CP0V4G01J type=write size=1431747 2024-03-04T17:50:33.288 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.289+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=WNK6097Z0FUO type=del size=1300001 2024-03-04T17:50:33.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.289+0000 7fda08af98c0 1 < canceled part key=_multipart_L0FVKWD7EQRW.2~XYN8HHSJOQDK.0 size=1048576 2024-03-04T17:50:33.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.293+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~XYN8HHSJOQDK.1 size=1048576 2024-03-04T17:50:33.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.297+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~XYN8HHSJOQDK.2 size=172388 2024-03-04T17:50:33.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.301+0000 7fda08af98c0 1 < canceled multipart upload key=L0FVKWD7EQRW upload=XYN8HHSJOQDK tag=D5UMTY2E55SO type=write size=3318116 2024-03-04T17:50:33.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.305+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=LBSD9W7D5F0P type=del size=2687448 2024-03-04T17:50:33.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.309+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=8ONDD1NAC4ID type=del 2024-03-04T17:50:33.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.317+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~JT7GTE50TW35.0 size=1048576 2024-03-04T17:50:33.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.321+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~JT7GTE50TW35.1 size=208695 2024-03-04T17:50:33.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.321+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=JT7GTE50TW35 tag=CNW0IPCLJTBR type=write size=2305847 2024-03-04T17:50:33.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.325+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~PBOYIP2MU66X.0 size=761191 2024-03-04T17:50:33.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.333+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=PBOYIP2MU66X tag=P24Y5GAWVVAU type=write size=1809767 2024-03-04T17:50:33.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.337+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~JNAWGRNAPJBM.0 size=41256 2024-03-04T17:50:33.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.337+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=JNAWGRNAPJBM tag=O3PFR9PY389S type=write size=1089832 2024-03-04T17:50:33.340 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.341+0000 7fda08af98c0 -1 < failed to complete operation key=XG2XYBCMUNY5 tag=J7G4Q2ZDQ1G2 type=del size=3351683: (22) Invalid argument 2024-03-04T17:50:33.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.345+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=5MRX79EM6LC4 type=del size=2529811 2024-03-04T17:50:33.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.349+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~PBNJOZ1VE0S0.0 size=1048576 2024-03-04T17:50:33.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.353+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~PBNJOZ1VE0S0.1 size=596744 2024-03-04T17:50:33.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.361+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=PBNJOZ1VE0S0 tag=HQNKYWRCJEGT type=write size=2693896 2024-03-04T17:50:33.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.369+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~293HICX35KEN.0 size=74338 2024-03-04T17:50:33.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.369+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=293HICX35KEN tag=0RVODVRLN9VN type=write size=1122914 2024-03-04T17:50:33.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.373+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~C987YR3AX98W.0 size=1048576 2024-03-04T17:50:33.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.373+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~C987YR3AX98W.1 size=1048576 2024-03-04T17:50:33.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.377+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~C987YR3AX98W.2 size=283818 2024-03-04T17:50:33.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.385+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=C987YR3AX98W tag=JEOLLQU8EJ3H type=write size=3429546 2024-03-04T17:50:33.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.393+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~S46JM7T0D6NS.0 size=1048576 2024-03-04T17:50:33.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.393+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~S46JM7T0D6NS.1 size=1048576 2024-03-04T17:50:33.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.397+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~S46JM7T0D6NS.2 size=871445 2024-03-04T17:50:33.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.397+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=S46JM7T0D6NS tag=9DAERIMSSNX6 type=write size=4017173 2024-03-04T17:50:33.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.401+0000 7fda08af98c0 1 < completed write operation key=09KBCA2XURTF tag=QWLMRRHL6I3R type=write size=603818 2024-03-04T17:50:33.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.409+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~J38NKECNQ4F4.0 size=1048576 2024-03-04T17:50:33.410 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.413+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~J38NKECNQ4F4.1 size=1048576 2024-03-04T17:50:33.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.413+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~J38NKECNQ4F4.2 size=1007746 2024-03-04T17:50:33.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.413+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=J38NKECNQ4F4 tag=HQQUOIYLQ2OL type=write size=4153474 2024-03-04T17:50:33.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.417+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=62ZGDCQRTH9T type=del 2024-03-04T17:50:33.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.425+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~NGJU0QCDEUY6.0 size=24032 2024-03-04T17:50:33.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.425+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=NGJU0QCDEUY6 tag=54J49ECCBOD7 type=write size=1072608 2024-03-04T17:50:33.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.429+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=67OULN1NBVLF type=write size=720692 2024-03-04T17:50:33.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.433+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=YJJ92CSH7L3A type=del size=4168872 2024-03-04T17:50:33.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.437+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=B390UTO38KP1 type=del 2024-03-04T17:50:33.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.445+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~NX15NB174TT6.0 size=1048576 2024-03-04T17:50:33.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.445+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~NX15NB174TT6.1 size=1048576 2024-03-04T17:50:33.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.449+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~NX15NB174TT6.2 size=856415 2024-03-04T17:50:33.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.449+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=NX15NB174TT6 tag=HBPIH1YVEFN3 type=write size=4002143 2024-03-04T17:50:33.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.453+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=SRCEKD0S0FA5 type=del 2024-03-04T17:50:33.458 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.461+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~3R6QYZV3E9JX.0 size=1048576 2024-03-04T17:50:33.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.461+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~3R6QYZV3E9JX.1 size=223421 2024-03-04T17:50:33.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.461+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=3R6QYZV3E9JX tag=60WHB1980C42 type=write size=2320573 2024-03-04T17:50:33.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.465+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=IART95URPGTP type=del 2024-03-04T17:50:33.469 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.469+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=M9CTEGH4GG93 type=del size=2393524 2024-03-04T17:50:33.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.473+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=4VP4BYTRY4WH type=del 2024-03-04T17:50:33.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.481+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~ROW5JQ9Y6807.0 size=1048576 2024-03-04T17:50:33.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.485+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~ROW5JQ9Y6807.1 size=647820 2024-03-04T17:50:33.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.485+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=ROW5JQ9Y6807 tag=QS2EMYTSCY4A type=write size=2744972 2024-03-04T17:50:33.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.489+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=DMCO7AA7KE2D type=del 2024-03-04T17:50:33.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.493+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~AYHRI1F7TX2O.0 size=7103 2024-03-04T17:50:33.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.493+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=AYHRI1F7TX2O tag=LAZF2WSB0G3D type=write size=1055679 2024-03-04T17:50:33.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.497+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=4QBMC8TU4FB5 type=del 2024-03-04T17:50:33.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.505+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~3GCC14Z45I7E.0 size=1048576 2024-03-04T17:50:33.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.509+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~3GCC14Z45I7E.1 size=727705 2024-03-04T17:50:33.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.509+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=3GCC14Z45I7E tag=CKL8NZOD53RO type=write size=2824857 2024-03-04T17:50:33.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.513+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=WNK6097Z0FUO type=del 2024-03-04T17:50:33.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.513+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=XUP56ZEVEBPF type=del size=1141866 2024-03-04T17:50:33.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.521+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=LBSD9W7D5F0P type=del 2024-03-04T17:50:33.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.525+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=LFHJHF2ZWCA8 type=del size=4130924 2024-03-04T17:50:33.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.529+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~JT7GTE50TW35.0 size=1048576 2024-03-04T17:50:33.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.529+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~JT7GTE50TW35.1 size=208695 2024-03-04T17:50:33.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.537+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=JT7GTE50TW35 tag=CNW0IPCLJTBR type=write size=2305847 2024-03-04T17:50:33.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.545+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~06IMNKYO16E2.0 size=1048576 2024-03-04T17:50:33.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.545+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~06IMNKYO16E2.1 size=667065 2024-03-04T17:50:33.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.545+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=06IMNKYO16E2 tag=SNHILQ7ERO8R type=write size=2764217 2024-03-04T17:50:33.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.549+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~JNAWGRNAPJBM.0 size=41256 2024-03-04T17:50:33.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.561+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=JNAWGRNAPJBM tag=O3PFR9PY389S type=write size=1089832 2024-03-04T17:50:33.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.565+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=87R7KUR5X3BN type=del size=3170854 2024-03-04T17:50:33.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.573+0000 7fda08af98c0 1 < suggested update operation key=3UWSL7NJLACY tag=5MRX79EM6LC4 type=del size=2529811 2024-03-04T17:50:33.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.577+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=1DF232OQOM5L type=del size=451365 2024-03-04T17:50:33.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.581+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~293HICX35KEN.0 size=74338 2024-03-04T17:50:33.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.585+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=293HICX35KEN tag=0RVODVRLN9VN type=write size=1122914 2024-03-04T17:50:33.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.589+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=HIG73ISYUSHL type=del size=1328624 2024-03-04T17:50:33.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.593+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~S46JM7T0D6NS.0 size=1048576 2024-03-04T17:50:33.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.597+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~S46JM7T0D6NS.1 size=1048576 2024-03-04T17:50:33.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.597+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~S46JM7T0D6NS.2 size=871445 2024-03-04T17:50:33.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.605+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=S46JM7T0D6NS tag=9DAERIMSSNX6 type=write size=4017173 2024-03-04T17:50:33.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.609+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=CYIWVVY4W923 type=del size=1366449 2024-03-04T17:50:33.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.613+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~J38NKECNQ4F4.0 size=1048576 2024-03-04T17:50:33.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.617+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~J38NKECNQ4F4.1 size=1048576 2024-03-04T17:50:33.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.617+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~J38NKECNQ4F4.2 size=1007746 2024-03-04T17:50:33.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.625+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=J38NKECNQ4F4 tag=HQQUOIYLQ2OL type=write size=4153474 2024-03-04T17:50:33.627 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.629+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=91WXH1AKAQRF type=del size=4065420 2024-03-04T17:50:33.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.633+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~NGJU0QCDEUY6.0 size=24032 2024-03-04T17:50:33.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.637+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=NGJU0QCDEUY6 tag=54J49ECCBOD7 type=write size=1072608 2024-03-04T17:50:33.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.641+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=BHNTGVGBH7QZ type=del size=2632535 2024-03-04T17:50:33.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.645+0000 7fda08af98c0 1 < suggested remove operation key=C1GK00FM2ZYA tag=YJJ92CSH7L3A type=del 2024-03-04T17:50:33.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.649+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=4OO4HGM4RKZV type=write size=363400 2024-03-04T17:50:33.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.653+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~NX15NB174TT6.0 size=1048576 2024-03-04T17:50:33.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.657+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~NX15NB174TT6.1 size=1048576 2024-03-04T17:50:33.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.657+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~NX15NB174TT6.2 size=856415 2024-03-04T17:50:33.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.661+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=NX15NB174TT6 tag=HBPIH1YVEFN3 type=write size=4002143 2024-03-04T17:50:33.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.669+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~WHA6EW3EW4IN.0 size=26908 2024-03-04T17:50:33.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.669+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=WHA6EW3EW4IN tag=BI4JOV0E7JDP type=write size=1075484 2024-03-04T17:50:33.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.673+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~3R6QYZV3E9JX.0 size=1048576 2024-03-04T17:50:33.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.673+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~3R6QYZV3E9JX.1 size=223421 2024-03-04T17:50:33.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.681+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=3R6QYZV3E9JX tag=60WHB1980C42 type=write size=2320573 2024-03-04T17:50:33.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.681+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=0UYWR1WEB1YU type=write size=887863 2024-03-04T17:50:33.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.685+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=M9CTEGH4GG93 type=del 2024-03-04T17:50:33.690 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.693+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~YL8TBN72B7S7.0 size=1048576 2024-03-04T17:50:33.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.693+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~YL8TBN72B7S7.1 size=919704 2024-03-04T17:50:33.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.693+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=YL8TBN72B7S7 tag=HJX70TEV05OL type=write size=3016856 2024-03-04T17:50:33.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.697+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~ROW5JQ9Y6807.0 size=1048576 2024-03-04T17:50:33.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.701+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~ROW5JQ9Y6807.1 size=647820 2024-03-04T17:50:33.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.705+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=ROW5JQ9Y6807 tag=QS2EMYTSCY4A type=write size=2744972 2024-03-04T17:50:33.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.709+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~P1ZPKIAGDP2U.0 size=318206 2024-03-04T17:50:33.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.709+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=P1ZPKIAGDP2U tag=UH539MB96QLB type=write size=1366782 2024-03-04T17:50:33.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.713+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~AYHRI1F7TX2O.0 size=7103 2024-03-04T17:50:33.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.721+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=AYHRI1F7TX2O tag=LAZF2WSB0G3D type=write size=1055679 2024-03-04T17:50:33.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.725+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=AO8G0JWMTLIQ type=del size=1007950 2024-03-04T17:50:33.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.725+0000 7fda08af98c0 1 < canceled part key=_multipart_G0BHMSCO0XLF.2~3GCC14Z45I7E.0 size=1048576 2024-03-04T17:50:33.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.729+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~3GCC14Z45I7E.1 size=727705 2024-03-04T17:50:33.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.733+0000 7fda08af98c0 1 < canceled multipart upload key=G0BHMSCO0XLF upload=3GCC14Z45I7E tag=CKL8NZOD53RO type=write size=2824857 2024-03-04T17:50:33.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.737+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~0RRCJRDOSTNJ.0 size=1048576 2024-03-04T17:50:33.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.741+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~0RRCJRDOSTNJ.1 size=462661 2024-03-04T17:50:33.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.741+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=0RRCJRDOSTNJ tag=JI4CDIOD2NNV type=write size=2559813 2024-03-04T17:50:33.744 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.745+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=XUP56ZEVEBPF type=del 2024-03-04T17:50:33.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.749+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=4XM74HT0M3XY type=del size=2621948 2024-03-04T17:50:33.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.753+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=LFHJHF2ZWCA8 type=del 2024-03-04T17:50:33.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.761+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~J622OU9OUZKU.0 size=54669 2024-03-04T17:50:33.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.761+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=J622OU9OUZKU tag=CLTO6QAYOZR1 type=write size=1103245 2024-03-04T17:50:33.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.765+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~06IMNKYO16E2.0 size=1048576 2024-03-04T17:50:33.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.769+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~06IMNKYO16E2.1 size=667065 2024-03-04T17:50:33.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.773+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=06IMNKYO16E2 tag=SNHILQ7ERO8R type=write size=2764217 2024-03-04T17:50:33.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.777+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=S57WXS3F4R0D type=del size=1533382 2024-03-04T17:50:33.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.781+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=87R7KUR5X3BN type=del 2024-03-04T17:50:33.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.785+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=EDFRZ56EEQIL type=write size=548426 2024-03-04T17:50:33.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.789+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=1DF232OQOM5L type=del 2024-03-04T17:50:33.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.793+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=59VLSXDV5KTY type=write size=680521 2024-03-04T17:50:33.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.797+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=HIG73ISYUSHL type=del 2024-03-04T17:50:33.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.805+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~NORTI0E73H86.0 size=602027 2024-03-04T17:50:33.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.805+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=NORTI0E73H86 tag=A0HFLZ2XG4FE type=write size=1650603 2024-03-04T17:50:33.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.813+0000 7fda08af98c0 1 < suggested remove operation key=0IGSMFORCSL9 tag=CYIWVVY4W923 type=del 2024-03-04T17:50:33.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.821+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~EQ3K3GAGA0GV.0 size=1048576 2024-03-04T17:50:33.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.821+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~EQ3K3GAGA0GV.1 size=1048576 2024-03-04T17:50:33.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.825+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~EQ3K3GAGA0GV.2 size=181613 2024-03-04T17:50:33.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.825+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=EQ3K3GAGA0GV tag=N373QP5JVQ7C type=write size=3327341 2024-03-04T17:50:33.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.829+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=91WXH1AKAQRF type=del 2024-03-04T17:50:33.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.837+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~OYW20EIHYYGP.0 size=1048576 2024-03-04T17:50:33.838 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.841+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~OYW20EIHYYGP.1 size=1048576 2024-03-04T17:50:33.841 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.841+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~OYW20EIHYYGP.2 size=129199 2024-03-04T17:50:33.841 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.841+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=OYW20EIHYYGP tag=ICZP45L00DWE type=write size=3274927 2024-03-04T17:50:33.844 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.845+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=BHNTGVGBH7QZ type=del 2024-03-04T17:50:33.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.853+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~DFRUHD8QJMUK.0 size=1048576 2024-03-04T17:50:33.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.857+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~DFRUHD8QJMUK.1 size=339255 2024-03-04T17:50:33.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.857+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=DFRUHD8QJMUK tag=8CAGCR1N8JQY type=write size=2436407 2024-03-04T17:50:33.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.861+0000 7fda08af98c0 1 < completed write operation key=HCA7QLGM2O4O tag=4OO4HGM4RKZV type=write size=363400 2024-03-04T17:50:33.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.865+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=E07WBLNANEVP type=del size=3418352 2024-03-04T17:50:33.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.869+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~WHA6EW3EW4IN.0 size=26908 2024-03-04T17:50:33.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.873+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=WHA6EW3EW4IN tag=BI4JOV0E7JDP type=write size=1075484 2024-03-04T17:50:33.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.881+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~BPATXHADCILE.0 size=1048576 2024-03-04T17:50:33.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.885+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~BPATXHADCILE.1 size=1048576 2024-03-04T17:50:33.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.889+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~BPATXHADCILE.2 size=728525 2024-03-04T17:50:33.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.889+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=BPATXHADCILE tag=Q6C3OCQKXUTP type=write size=3874253 2024-03-04T17:50:33.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.897+0000 7fda08af98c0 1 < suggested remove operation key=JA4JK99I0942 tag=0UYWR1WEB1YU type=write 2024-03-04T17:50:33.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.901+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=APKN3VOXDTD4 type=del size=3273705 2024-03-04T17:50:33.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.905+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~YL8TBN72B7S7.0 size=1048576 2024-03-04T17:50:33.905 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.905+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~YL8TBN72B7S7.1 size=919704 2024-03-04T17:50:33.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.913+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=YL8TBN72B7S7 tag=HJX70TEV05OL type=write size=3016856 2024-03-04T17:50:33.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.921+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~TAOU2LI7ZZXZ.0 size=1048576 2024-03-04T17:50:33.921 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.921+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~TAOU2LI7ZZXZ.1 size=234225 2024-03-04T17:50:33.921 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.921+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=TAOU2LI7ZZXZ tag=38GG0R6IEZHT type=write size=2331377 2024-03-04T17:50:33.925 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.925+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~P1ZPKIAGDP2U.0 size=318206 2024-03-04T17:50:33.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.933+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=P1ZPKIAGDP2U tag=UH539MB96QLB type=write size=1366782 2024-03-04T17:50:33.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.937+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=18C505CWMAWW type=del size=2052129 2024-03-04T17:50:33.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.941+0000 7fda08af98c0 1 < canceled operation key=V1LDB2HXZ7EF tag=AO8G0JWMTLIQ type=del size=1007950 2024-03-04T17:50:33.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.945+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~SVFPR11Q88K8.0 size=1044799 2024-03-04T17:50:33.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.945+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=SVFPR11Q88K8 tag=O4ET6DUM6UHL type=write size=2093375 2024-03-04T17:50:33.949 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.949+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~0RRCJRDOSTNJ.0 size=1048576 2024-03-04T17:50:33.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.953+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~0RRCJRDOSTNJ.1 size=462661 2024-03-04T17:50:33.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.961+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=0RRCJRDOSTNJ tag=JI4CDIOD2NNV type=write size=2559813 2024-03-04T17:50:33.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.965+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~MRE7T44LCF91.0 size=1048576 2024-03-04T17:50:33.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.969+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~MRE7T44LCF91.1 size=1048576 2024-03-04T17:50:33.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.973+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~MRE7T44LCF91.2 size=33758 2024-03-04T17:50:33.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.973+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=MRE7T44LCF91 tag=TLX25C5HZD2F type=write size=3179486 2024-03-04T17:50:33.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.977+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=4XM74HT0M3XY type=del 2024-03-04T17:50:33.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.981+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~QIHCPMC9K04Y.0 size=1048576 2024-03-04T17:50:33.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.985+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~QIHCPMC9K04Y.1 size=1048576 2024-03-04T17:50:33.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.989+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~QIHCPMC9K04Y.2 size=759049 2024-03-04T17:50:33.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.989+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=QIHCPMC9K04Y tag=XGO156AOCV4X type=write size=3904777 2024-03-04T17:50:33.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.993+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~J622OU9OUZKU.0 size=54669 2024-03-04T17:50:33.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:33.997+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=J622OU9OUZKU tag=CLTO6QAYOZR1 type=write size=1103245 2024-03-04T17:50:34.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.005+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=U68Q9G9Y62XF type=write size=959318 2024-03-04T17:50:34.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.009+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=S57WXS3F4R0D type=del 2024-03-04T17:50:34.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.013+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=TCI4J1YMJUJS type=write size=736857 2024-03-04T17:50:34.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.017+0000 7fda08af98c0 1 < completed write operation key=HCKJZUMAK0Z0 tag=EDFRZ56EEQIL type=write size=548426 2024-03-04T17:50:34.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.021+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=3ASGWY2F7XDL type=write size=147506 2024-03-04T17:50:34.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.025+0000 7fda08af98c0 1 < completed write operation key=IJSYAVR5SWPS tag=59VLSXDV5KTY type=write size=680521 2024-03-04T17:50:34.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.029+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~D8RPZXF2UVL9.0 size=1048576 2024-03-04T17:50:34.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.033+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~D8RPZXF2UVL9.1 size=666355 2024-03-04T17:50:34.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.033+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=D8RPZXF2UVL9 tag=9FXZKT2EM290 type=write size=2763507 2024-03-04T17:50:34.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.033+0000 7fda08af98c0 1 < canceled part key=_multipart_X5MEJ7G8X9HE.2~NORTI0E73H86.0 size=602027 2024-03-04T17:50:34.039 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.041+0000 7fda08af98c0 1 < canceled multipart upload key=X5MEJ7G8X9HE upload=NORTI0E73H86 tag=A0HFLZ2XG4FE type=write size=1650603 2024-03-04T17:50:34.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.049+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~ZW9TK19GR19S.0 size=1048576 2024-03-04T17:50:34.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.049+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~ZW9TK19GR19S.1 size=954767 2024-03-04T17:50:34.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.049+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=ZW9TK19GR19S tag=IU16ZZWDK7E1 type=write size=3051919 2024-03-04T17:50:34.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.053+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~EQ3K3GAGA0GV.0 size=1048576 2024-03-04T17:50:34.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.057+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~EQ3K3GAGA0GV.1 size=1048576 2024-03-04T17:50:34.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.057+0000 7fda08af98c0 1 < canceled part key=_multipart_GDGTJZ21TG32.2~EQ3K3GAGA0GV.2 size=181613 2024-03-04T17:50:34.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.065+0000 7fda08af98c0 1 < canceled multipart upload key=GDGTJZ21TG32 upload=EQ3K3GAGA0GV tag=N373QP5JVQ7C type=write size=3327341 2024-03-04T17:50:34.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.069+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=0LFZ291K59IU type=write size=313999 2024-03-04T17:50:34.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.073+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~OYW20EIHYYGP.0 size=1048576 2024-03-04T17:50:34.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.073+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~OYW20EIHYYGP.1 size=1048576 2024-03-04T17:50:34.076 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.077+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~OYW20EIHYYGP.2 size=129199 2024-03-04T17:50:34.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.085+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=OYW20EIHYYGP tag=ICZP45L00DWE type=write size=3274927 2024-03-04T17:50:34.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.089+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=K48I9X60G86T type=del size=806729 2024-03-04T17:50:34.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.093+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~DFRUHD8QJMUK.0 size=1048576 2024-03-04T17:50:34.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.093+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~DFRUHD8QJMUK.1 size=339255 2024-03-04T17:50:34.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.101+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=DFRUHD8QJMUK tag=8CAGCR1N8JQY type=write size=2436407 2024-03-04T17:50:34.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.109+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~YXZ6X9WXPDCZ.0 size=1048576 2024-03-04T17:50:34.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.109+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~YXZ6X9WXPDCZ.1 size=1048576 2024-03-04T17:50:34.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.113+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~YXZ6X9WXPDCZ.2 size=1026705 2024-03-04T17:50:34.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.113+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=YXZ6X9WXPDCZ tag=VPSX73F83VWF type=write size=4172433 2024-03-04T17:50:34.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.117+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=E07WBLNANEVP type=del 2024-03-04T17:50:34.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.125+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~XCWIY37F5LYQ.0 size=1048576 2024-03-04T17:50:34.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.125+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~XCWIY37F5LYQ.1 size=1048576 2024-03-04T17:50:34.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.129+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~XCWIY37F5LYQ.2 size=696956 2024-03-04T17:50:34.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.129+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=XCWIY37F5LYQ tag=GY2GANEVOWFO type=write size=3842684 2024-03-04T17:50:34.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.133+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~BPATXHADCILE.0 size=1048576 2024-03-04T17:50:34.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.137+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~BPATXHADCILE.1 size=1048576 2024-03-04T17:50:34.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.141+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~BPATXHADCILE.2 size=728525 2024-03-04T17:50:34.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.149+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=BPATXHADCILE tag=Q6C3OCQKXUTP type=write size=3874253 2024-03-04T17:50:34.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.157+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~S1TX65TZ24BU.0 size=280922 2024-03-04T17:50:34.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.157+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=S1TX65TZ24BU tag=NWGWF19CYEDJ type=write size=1329498 2024-03-04T17:50:34.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.161+0000 7fda08af98c0 1 < canceled operation key=XG5I3SDRE5JM tag=APKN3VOXDTD4 type=del size=3273705 2024-03-04T17:50:34.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.165+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=Q91CNCRJ5WR2 type=write size=1017141 2024-03-04T17:50:34.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.169+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~TAOU2LI7ZZXZ.0 size=1048576 2024-03-04T17:50:34.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.173+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~TAOU2LI7ZZXZ.1 size=234225 2024-03-04T17:50:34.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.181+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=TAOU2LI7ZZXZ tag=38GG0R6IEZHT type=write size=2331377 2024-03-04T17:50:34.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.189+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~HR1Y3B9DUD90.0 size=819513 2024-03-04T17:50:34.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.189+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=HR1Y3B9DUD90 tag=TLR8LI3I3TUA type=write size=1868089 2024-03-04T17:50:34.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.193+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=18C505CWMAWW type=del 2024-03-04T17:50:34.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.201+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~AOZ75MGOJMVX.0 size=278190 2024-03-04T17:50:34.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.201+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=AOZ75MGOJMVX tag=DELPC6JHEGU6 type=write size=1326766 2024-03-04T17:50:34.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.201+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~SVFPR11Q88K8.0 size=1044799 2024-03-04T17:50:34.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.209+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=SVFPR11Q88K8 tag=O4ET6DUM6UHL type=write size=2093375 2024-03-04T17:50:34.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.213+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=S4X923I5TAF8 type=del size=2574943 2024-03-04T17:50:34.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.217+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~MRE7T44LCF91.0 size=1048576 2024-03-04T17:50:34.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.221+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~MRE7T44LCF91.1 size=1048576 2024-03-04T17:50:34.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.221+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~MRE7T44LCF91.2 size=33758 2024-03-04T17:50:34.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.229+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=MRE7T44LCF91 tag=TLX25C5HZD2F type=write size=3179486 2024-03-04T17:50:34.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.237+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~LZH5T3ZUO0XR.0 size=772569 2024-03-04T17:50:34.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.237+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=LZH5T3ZUO0XR tag=7I3LHU7495RE type=write size=1821145 2024-03-04T17:50:34.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.241+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~QIHCPMC9K04Y.0 size=1048576 2024-03-04T17:50:34.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.245+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~QIHCPMC9K04Y.1 size=1048576 2024-03-04T17:50:34.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.249+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~QIHCPMC9K04Y.2 size=759049 2024-03-04T17:50:34.254 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.257+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=QIHCPMC9K04Y tag=XGO156AOCV4X type=write size=3904777 2024-03-04T17:50:34.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.261+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~NK1G8RAVJKOZ.0 size=555388 2024-03-04T17:50:34.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.261+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=NK1G8RAVJKOZ tag=LV0K8PYB7YAL type=write size=1603964 2024-03-04T17:50:34.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.265+0000 7fda08af98c0 1 < completed write operation key=L0FVKWD7EQRW tag=U68Q9G9Y62XF type=write size=959318 2024-03-04T17:50:34.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.269+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=7U8D9H4UY5OE type=del size=1401710 2024-03-04T17:50:34.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.277+0000 7fda08af98c0 1 < completed write operation key=3UWSL7NJLACY tag=TCI4J1YMJUJS type=write size=736857 2024-03-04T17:50:34.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.281+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=XYBIUEJX6AGI type=del size=2831227 2024-03-04T17:50:34.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.285+0000 7fda08af98c0 1 < completed write operation key=VJVORUG95FLC tag=3ASGWY2F7XDL type=write size=147506 2024-03-04T17:50:34.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.293+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~343W9A8FXGXB.0 size=663411 2024-03-04T17:50:34.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.293+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=343W9A8FXGXB tag=ZENKVXPRQDL1 type=write size=1711987 2024-03-04T17:50:34.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.297+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~D8RPZXF2UVL9.0 size=1048576 2024-03-04T17:50:34.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.301+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~D8RPZXF2UVL9.1 size=666355 2024-03-04T17:50:34.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.305+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=D8RPZXF2UVL9 tag=9FXZKT2EM290 type=write size=2763507 2024-03-04T17:50:34.308 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.309+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=TBFVY4WH59QF type=write size=283261 2024-03-04T17:50:34.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.313+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~ZW9TK19GR19S.0 size=1048576 2024-03-04T17:50:34.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.317+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~ZW9TK19GR19S.1 size=954767 2024-03-04T17:50:34.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.321+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=ZW9TK19GR19S tag=IU16ZZWDK7E1 type=write size=3051919 2024-03-04T17:50:34.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.329+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~MHXQF0SYID7P.0 size=1033825 2024-03-04T17:50:34.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.329+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=MHXQF0SYID7P tag=D68E4RT80XKO type=write size=2082401 2024-03-04T17:50:34.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.333+0000 7fda08af98c0 1 < canceled operation key=1OZKPHMPV3OQ tag=0LFZ291K59IU type=write size=313999 2024-03-04T17:50:34.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.341+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~3XSI7BQACI1Y.0 size=1048576 2024-03-04T17:50:34.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.345+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~3XSI7BQACI1Y.1 size=827388 2024-03-04T17:50:34.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.345+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=3XSI7BQACI1Y tag=008JHIUSTQ99 type=write size=2924540 2024-03-04T17:50:34.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.349+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=K48I9X60G86T type=del size=806729 2024-03-04T17:50:34.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.353+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=WGV7OD4P34JW type=del size=3258047 2024-03-04T17:50:34.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.357+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~YXZ6X9WXPDCZ.0 size=1048576 2024-03-04T17:50:34.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.357+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~YXZ6X9WXPDCZ.1 size=1048576 2024-03-04T17:50:34.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.361+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~YXZ6X9WXPDCZ.2 size=1026705 2024-03-04T17:50:34.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.365+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=YXZ6X9WXPDCZ tag=VPSX73F83VWF type=write size=4172433 2024-03-04T17:50:34.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.373+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~CKN6JL1YAZL2.0 size=176779 2024-03-04T17:50:34.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.373+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=CKN6JL1YAZL2 tag=HWMNZK5WNVM1 type=write size=1225355 2024-03-04T17:50:34.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.377+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~XCWIY37F5LYQ.0 size=1048576 2024-03-04T17:50:34.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.381+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~XCWIY37F5LYQ.1 size=1048576 2024-03-04T17:50:34.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.385+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~XCWIY37F5LYQ.2 size=696956 2024-03-04T17:50:34.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.389+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=XCWIY37F5LYQ tag=GY2GANEVOWFO type=write size=3842684 2024-03-04T17:50:34.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.393+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=02C03WQK32CF type=del size=3734244 2024-03-04T17:50:34.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.397+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~S1TX65TZ24BU.0 size=280922 2024-03-04T17:50:34.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.405+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=S1TX65TZ24BU tag=NWGWF19CYEDJ type=write size=1329498 2024-03-04T17:50:34.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.409+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=N5RXQ0EK8IPO type=del size=1456987 2024-03-04T17:50:34.410 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.413+0000 7fda08af98c0 1 < completed write operation key=Z2JQBN84QQIB tag=Q91CNCRJ5WR2 type=write size=1017141 2024-03-04T17:50:34.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.417+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~2J6CMCMTFXAG.0 size=1048576 2024-03-04T17:50:34.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.421+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~2J6CMCMTFXAG.1 size=599531 2024-03-04T17:50:34.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.421+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=2J6CMCMTFXAG tag=J6827WLDKPKQ type=write size=2696683 2024-03-04T17:50:34.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.425+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~HR1Y3B9DUD90.0 size=819513 2024-03-04T17:50:34.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.433+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=HR1Y3B9DUD90 tag=TLR8LI3I3TUA type=write size=1868089 2024-03-04T17:50:34.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.433+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=0MN12PL2B2FL type=del size=356884 2024-03-04T17:50:34.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.437+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~AOZ75MGOJMVX.0 size=278190 2024-03-04T17:50:34.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.445+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=AOZ75MGOJMVX tag=DELPC6JHEGU6 type=write size=1326766 2024-03-04T17:50:34.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.453+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~JZ7XXLZQ4HG2.0 size=1048576 2024-03-04T17:50:34.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.453+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~JZ7XXLZQ4HG2.1 size=1048576 2024-03-04T17:50:34.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.457+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~JZ7XXLZQ4HG2.2 size=84398 2024-03-04T17:50:34.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.457+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=JZ7XXLZQ4HG2 tag=R1M88TIJC3AW type=write size=3230126 2024-03-04T17:50:34.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.465+0000 7fda08af98c0 1 < suggested update operation key=Z3J7W9V8HQKH tag=S4X923I5TAF8 type=del size=2574943 2024-03-04T17:50:34.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.473+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~5ENEHUX1W7JV.0 size=35460 2024-03-04T17:50:34.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.473+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=5ENEHUX1W7JV tag=2M0O614XXS8N type=write size=1084036 2024-03-04T17:50:34.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.477+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~LZH5T3ZUO0XR.0 size=772569 2024-03-04T17:50:34.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.481+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=LZH5T3ZUO0XR tag=7I3LHU7495RE type=write size=1821145 2024-03-04T17:50:34.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.485+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=P0L8HKW2GSMJ type=del size=1652200 2024-03-04T17:50:34.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.489+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~NK1G8RAVJKOZ.0 size=555388 2024-03-04T17:50:34.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.497+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=NK1G8RAVJKOZ tag=LV0K8PYB7YAL type=write size=1603964 2024-03-04T17:50:34.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.505+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~VN7R5QBA0I8R.0 size=1048576 2024-03-04T17:50:34.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.505+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~VN7R5QBA0I8R.1 size=327600 2024-03-04T17:50:34.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.505+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=VN7R5QBA0I8R tag=1U21C8BFNWH8 type=write size=2424752 2024-03-04T17:50:34.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.513+0000 7fda08af98c0 1 < suggested remove operation key=FMPZ2548LBBF tag=7U8D9H4UY5OE type=del 2024-03-04T17:50:34.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.521+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~66LPTJYIJ135.0 size=1048576 2024-03-04T17:50:34.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.525+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~66LPTJYIJ135.1 size=1048576 2024-03-04T17:50:34.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.525+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~66LPTJYIJ135.2 size=670375 2024-03-04T17:50:34.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.525+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=66LPTJYIJ135 tag=7SDY054X2VJP type=write size=3816103 2024-03-04T17:50:34.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.529+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=XYBIUEJX6AGI type=del 2024-03-04T17:50:34.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.533+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=3S6IEVAIWWBY type=del size=3875036 2024-03-04T17:50:34.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.537+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~343W9A8FXGXB.0 size=663411 2024-03-04T17:50:34.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.541+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=343W9A8FXGXB tag=ZENKVXPRQDL1 type=write size=1711987 2024-03-04T17:50:34.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.545+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=H5IXOSUJ5RRP type=del size=3966895 2024-03-04T17:50:34.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.549+0000 7fda08af98c0 1 < completed write operation key=AO3QS6CM7I5D tag=TBFVY4WH59QF type=write size=283261 2024-03-04T17:50:34.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.553+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=LMGRERW16OIG type=del size=1779643 2024-03-04T17:50:34.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.557+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~MHXQF0SYID7P.0 size=1033825 2024-03-04T17:50:34.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.565+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=MHXQF0SYID7P tag=D68E4RT80XKO type=write size=2082401 2024-03-04T17:50:34.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.569+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=WNKWO913E3WK type=del size=848330 2024-03-04T17:50:34.573 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.573+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~3XSI7BQACI1Y.0 size=1048576 2024-03-04T17:50:34.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.577+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~3XSI7BQACI1Y.1 size=827388 2024-03-04T17:50:34.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.585+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=3XSI7BQACI1Y tag=008JHIUSTQ99 type=write size=2924540 2024-03-04T17:50:34.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.593+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~WTSZ71M9ZN42.0 size=1048576 2024-03-04T17:50:34.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.597+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~WTSZ71M9ZN42.1 size=1048576 2024-03-04T17:50:34.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.601+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~WTSZ71M9ZN42.2 size=275375 2024-03-04T17:50:34.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.601+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=WTSZ71M9ZN42 tag=09DRD3ONI4AM type=write size=3421103 2024-03-04T17:50:34.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.605+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=WGV7OD4P34JW type=del 2024-03-04T17:50:34.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.613+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~USCAZZRA3LKO.0 size=1048576 2024-03-04T17:50:34.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.613+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~USCAZZRA3LKO.1 size=1048576 2024-03-04T17:50:34.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.617+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~USCAZZRA3LKO.2 size=802168 2024-03-04T17:50:34.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.617+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=USCAZZRA3LKO tag=UWISXJZ6VNCD type=write size=3947896 2024-03-04T17:50:34.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.621+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~CKN6JL1YAZL2.0 size=176779 2024-03-04T17:50:34.627 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.629+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=CKN6JL1YAZL2 tag=HWMNZK5WNVM1 type=write size=1225355 2024-03-04T17:50:34.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.633+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=ILYUN1E4VRAJ type=write size=946124 2024-03-04T17:50:34.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.637+0000 7fda08af98c0 -1 < failed to complete operation key=FMPZ2548LBBF tag=02C03WQK32CF type=del size=3734244: (22) Invalid argument 2024-03-04T17:50:34.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.645+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~TSR95PHF0GUX.0 size=1048576 2024-03-04T17:50:34.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.645+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~TSR95PHF0GUX.1 size=462278 2024-03-04T17:50:34.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.645+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=TSR95PHF0GUX tag=50KQE6PX1L9Q type=write size=2559430 2024-03-04T17:50:34.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.649+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=N5RXQ0EK8IPO type=del 2024-03-04T17:50:34.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.653+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=1GW02DAW0PR4 type=del size=2098585 2024-03-04T17:50:34.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.657+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~2J6CMCMTFXAG.0 size=1048576 2024-03-04T17:50:34.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.657+0000 7fda08af98c0 1 < canceled part key=_multipart_09KBCA2XURTF.2~2J6CMCMTFXAG.1 size=599531 2024-03-04T17:50:34.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.665+0000 7fda08af98c0 1 < canceled multipart upload key=09KBCA2XURTF upload=2J6CMCMTFXAG tag=J6827WLDKPKQ type=write size=2696683 2024-03-04T17:50:34.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.669+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=OWC6NM0NTF0J type=del size=533200 2024-03-04T17:50:34.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.673+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=0MN12PL2B2FL type=del 2024-03-04T17:50:34.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.677+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=UDB1P6UCFLF0 type=del size=3250311 2024-03-04T17:50:34.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.681+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~JZ7XXLZQ4HG2.0 size=1048576 2024-03-04T17:50:34.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.681+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~JZ7XXLZQ4HG2.1 size=1048576 2024-03-04T17:50:34.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.685+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~JZ7XXLZQ4HG2.2 size=84398 2024-03-04T17:50:34.690 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.693+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=JZ7XXLZQ4HG2 tag=R1M88TIJC3AW type=write size=3230126 2024-03-04T17:50:34.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.697+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=JX9WI2FFD7WS type=del size=3390659 2024-03-04T17:50:34.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.701+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~5ENEHUX1W7JV.0 size=35460 2024-03-04T17:50:34.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.705+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=5ENEHUX1W7JV tag=2M0O614XXS8N type=write size=1084036 2024-03-04T17:50:34.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.713+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~U0QQZWDC5SCP.0 size=1048576 2024-03-04T17:50:34.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.717+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~U0QQZWDC5SCP.1 size=469975 2024-03-04T17:50:34.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.717+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=U0QQZWDC5SCP tag=PNRS4FDASX8N type=write size=2567127 2024-03-04T17:50:34.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.721+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=P0L8HKW2GSMJ type=del 2024-03-04T17:50:34.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.725+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=H9BL2730BZMP type=del size=833130 2024-03-04T17:50:34.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.729+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~VN7R5QBA0I8R.0 size=1048576 2024-03-04T17:50:34.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.733+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~VN7R5QBA0I8R.1 size=327600 2024-03-04T17:50:34.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.737+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=VN7R5QBA0I8R tag=1U21C8BFNWH8 type=write size=2424752 2024-03-04T17:50:34.744 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.745+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~GR9K1TDZ4KOQ.0 size=1048576 2024-03-04T17:50:34.747 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.749+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~GR9K1TDZ4KOQ.1 size=1048576 2024-03-04T17:50:34.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.753+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~GR9K1TDZ4KOQ.2 size=7472 2024-03-04T17:50:34.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.753+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=GR9K1TDZ4KOQ tag=XGTIGBNVXBWI type=write size=3153200 2024-03-04T17:50:34.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.757+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~66LPTJYIJ135.0 size=1048576 2024-03-04T17:50:34.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.757+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~66LPTJYIJ135.1 size=1048576 2024-03-04T17:50:34.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.757+0000 7fda08af98c0 1 < canceled part key=_multipart_V1LDB2HXZ7EF.2~66LPTJYIJ135.2 size=670375 2024-03-04T17:50:34.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.765+0000 7fda08af98c0 1 < canceled multipart upload key=V1LDB2HXZ7EF upload=66LPTJYIJ135 tag=7SDY054X2VJP type=write size=3816103 2024-03-04T17:50:34.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.769+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=4QC2TBMQZ878 type=del size=3952092 2024-03-04T17:50:34.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.773+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=3S6IEVAIWWBY type=del 2024-03-04T17:50:34.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.777+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=MMOZ7KT8HZOV type=write size=1026952 2024-03-04T17:50:34.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.781+0000 7fda08af98c0 1 < canceled operation key=1OZKPHMPV3OQ tag=H5IXOSUJ5RRP type=del size=3966895 2024-03-04T17:50:34.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.785+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=IEUBMEG52I32 type=del size=2280030 2024-03-04T17:50:34.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.789+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=LMGRERW16OIG type=del 2024-03-04T17:50:34.789 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.789+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=0MRKK6RSFOGD type=del size=2128881 2024-03-04T17:50:34.793 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.793+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=WNKWO913E3WK type=del 2024-03-04T17:50:34.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.801+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~YZKSP6Y68AO1.0 size=417960 2024-03-04T17:50:34.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.801+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=YZKSP6Y68AO1 tag=DO619EWB4AI7 type=write size=1466536 2024-03-04T17:50:34.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.801+0000 7fda08af98c0 1 < canceled part key=_multipart_IJSYAVR5SWPS.2~WTSZ71M9ZN42.0 size=1048576 2024-03-04T17:50:34.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.805+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~WTSZ71M9ZN42.1 size=1048576 2024-03-04T17:50:34.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.809+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~WTSZ71M9ZN42.2 size=275375 2024-03-04T17:50:34.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.817+0000 7fda08af98c0 1 < canceled multipart upload key=IJSYAVR5SWPS upload=WTSZ71M9ZN42 tag=09DRD3ONI4AM type=write size=3421103 2024-03-04T17:50:34.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.817+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=02OQSG4SOKWO type=del size=3879370 2024-03-04T17:50:34.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.821+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~USCAZZRA3LKO.0 size=1048576 2024-03-04T17:50:34.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.825+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~USCAZZRA3LKO.1 size=1048576 2024-03-04T17:50:34.827 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.829+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~USCAZZRA3LKO.2 size=802168 2024-03-04T17:50:34.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.833+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=USCAZZRA3LKO tag=UWISXJZ6VNCD type=write size=3947896 2024-03-04T17:50:34.840 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.841+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~2G8GJNNJIJDS.0 size=1048576 2024-03-04T17:50:34.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.845+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~2G8GJNNJIJDS.1 size=1048576 2024-03-04T17:50:34.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.849+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~2G8GJNNJIJDS.2 size=851634 2024-03-04T17:50:34.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.849+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=2G8GJNNJIJDS tag=7Q7X0X9XZC5Q type=write size=3997362 2024-03-04T17:50:34.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.853+0000 7fda08af98c0 1 < completed write operation key=L0FVKWD7EQRW tag=ILYUN1E4VRAJ type=write size=946124 2024-03-04T17:50:34.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.857+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=MMYFOC6L3SZL type=del size=580969 2024-03-04T17:50:34.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.861+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~TSR95PHF0GUX.0 size=1048576 2024-03-04T17:50:34.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.865+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~TSR95PHF0GUX.1 size=462278 2024-03-04T17:50:34.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.869+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=TSR95PHF0GUX tag=50KQE6PX1L9Q type=write size=2559430 2024-03-04T17:50:34.875 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.877+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~MAQGUMVMP5TA.0 size=1048576 2024-03-04T17:50:34.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.881+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~MAQGUMVMP5TA.1 size=248632 2024-03-04T17:50:34.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.881+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=MAQGUMVMP5TA tag=DFSL2TPPF0XS type=write size=2345784 2024-03-04T17:50:34.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.885+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=1GW02DAW0PR4 type=del 2024-03-04T17:50:34.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.889+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~PCZW3S5M5ZIE.0 size=1048576 2024-03-04T17:50:34.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.893+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~PCZW3S5M5ZIE.1 size=1034809 2024-03-04T17:50:34.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.893+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=PCZW3S5M5ZIE tag=ABGR0AV969CH type=write size=3131961 2024-03-04T17:50:34.896 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.897+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=OWC6NM0NTF0J type=del 2024-03-04T17:50:34.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.901+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=FW2B1ZOQXWX8 type=del size=2253461 2024-03-04T17:50:34.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.909+0000 7fda08af98c0 1 < suggested remove operation key=X5MEJ7G8X9HE tag=UDB1P6UCFLF0 type=del 2024-03-04T17:50:34.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.913+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=Q3H46HA75IEC type=del size=3871149 2024-03-04T17:50:34.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.917+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=JX9WI2FFD7WS type=del 2024-03-04T17:50:34.921 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.921+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~8OTW3RL1QZSE.0 size=244160 2024-03-04T17:50:34.921 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.921+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=8OTW3RL1QZSE tag=NULB46FGG1XL type=write size=1292736 2024-03-04T17:50:34.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.925+0000 7fda08af98c0 1 < canceled part key=_multipart_XG5I3SDRE5JM.2~U0QQZWDC5SCP.0 size=1048576 2024-03-04T17:50:34.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.925+0000 7fda08af98c0 1 < canceled part key=_multipart_XG5I3SDRE5JM.2~U0QQZWDC5SCP.1 size=469975 2024-03-04T17:50:34.928 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.929+0000 7fda08af98c0 1 < canceled multipart upload key=XG5I3SDRE5JM upload=U0QQZWDC5SCP tag=PNRS4FDASX8N type=write size=2567127 2024-03-04T17:50:34.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.933+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=DN81X2M6ANSY type=del size=4136001 2024-03-04T17:50:34.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.937+0000 7fda08af98c0 1 < canceled operation key=HCA7QLGM2O4O tag=H9BL2730BZMP type=del size=833130 2024-03-04T17:50:34.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.941+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=CIUUXXYH64LO type=del size=1208825 2024-03-04T17:50:34.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.945+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~GR9K1TDZ4KOQ.0 size=1048576 2024-03-04T17:50:34.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.949+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~GR9K1TDZ4KOQ.1 size=1048576 2024-03-04T17:50:34.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.949+0000 7fda08af98c0 1 < canceled part key=_multipart_IJSYAVR5SWPS.2~GR9K1TDZ4KOQ.2 size=7472 2024-03-04T17:50:34.954 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.957+0000 7fda08af98c0 1 < canceled multipart upload key=IJSYAVR5SWPS upload=GR9K1TDZ4KOQ tag=XGTIGBNVXBWI type=write size=3153200 2024-03-04T17:50:34.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.961+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=ARBU8RRU0G36 type=del size=1592189 2024-03-04T17:50:34.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.965+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=4QC2TBMQZ878 type=del 2024-03-04T17:50:34.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.965+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=RAQKAEPN59YA type=write size=759953 2024-03-04T17:50:34.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.973+0000 7fda08af98c0 1 < suggested remove operation key=C4B45GL40IMZ tag=MMOZ7KT8HZOV type=write 2024-03-04T17:50:34.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.981+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~W1UWVBCUJJ60.0 size=646990 2024-03-04T17:50:34.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.981+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=W1UWVBCUJJ60 tag=IRVLB08K78TK type=write size=1695566 2024-03-04T17:50:34.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.985+0000 7fda08af98c0 1 < canceled operation key=750NSVFBS03A tag=IEUBMEG52I32 type=del size=2280030 2024-03-04T17:50:34.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.989+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=OK2JA2L09PO1 type=del size=2039034 2024-03-04T17:50:34.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.993+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=0MRKK6RSFOGD type=del 2024-03-04T17:50:34.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:34.997+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=IJR5VQXV0FGR type=del size=1884190 2024-03-04T17:50:35.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.001+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~YZKSP6Y68AO1.0 size=417960 2024-03-04T17:50:35.007 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.009+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=YZKSP6Y68AO1 tag=DO619EWB4AI7 type=write size=1466536 2024-03-04T17:50:35.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.013+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~PQZ8M42T2Y6K.0 size=1048576 2024-03-04T17:50:35.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.017+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~PQZ8M42T2Y6K.1 size=1048576 2024-03-04T17:50:35.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.021+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~PQZ8M42T2Y6K.2 size=195757 2024-03-04T17:50:35.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.021+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=PQZ8M42T2Y6K tag=Q2MVNT19XVR5 type=write size=3341485 2024-03-04T17:50:35.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.025+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=02OQSG4SOKWO type=del 2024-03-04T17:50:35.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.033+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~CHR6UOC7XJZ8.0 size=1048576 2024-03-04T17:50:35.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.037+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~CHR6UOC7XJZ8.1 size=1048576 2024-03-04T17:50:35.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.037+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~CHR6UOC7XJZ8.2 size=227344 2024-03-04T17:50:35.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.037+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=CHR6UOC7XJZ8 tag=V3WL89P84Q54 type=write size=3373072 2024-03-04T17:50:35.040 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.041+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~2G8GJNNJIJDS.0 size=1048576 2024-03-04T17:50:35.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.045+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~2G8GJNNJIJDS.1 size=1048576 2024-03-04T17:50:35.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.049+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~2G8GJNNJIJDS.2 size=851634 2024-03-04T17:50:35.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.053+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=2G8GJNNJIJDS tag=7Q7X0X9XZC5Q type=write size=3997362 2024-03-04T17:50:35.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.061+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~ABY381LQW3AA.0 size=1048576 2024-03-04T17:50:35.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.065+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~ABY381LQW3AA.1 size=201782 2024-03-04T17:50:35.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.065+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=ABY381LQW3AA tag=8N6G4KJ9P6TQ type=write size=2298934 2024-03-04T17:50:35.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.069+0000 7fda08af98c0 1 < canceled operation key=BYQFZSJ3WH07 tag=MMYFOC6L3SZL type=del size=580969 2024-03-04T17:50:35.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.073+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~2O0ASKH49ZXQ.0 size=395872 2024-03-04T17:50:35.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.073+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=2O0ASKH49ZXQ tag=8YLHQJX0UHOC type=write size=1444448 2024-03-04T17:50:35.077 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.077+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~MAQGUMVMP5TA.0 size=1048576 2024-03-04T17:50:35.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.081+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~MAQGUMVMP5TA.1 size=248632 2024-03-04T17:50:35.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.089+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=MAQGUMVMP5TA tag=DFSL2TPPF0XS type=write size=2345784 2024-03-04T17:50:35.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.093+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=O4GFPHSUGZ01 type=del size=2316276 2024-03-04T17:50:35.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.097+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~PCZW3S5M5ZIE.0 size=1048576 2024-03-04T17:50:35.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.097+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~PCZW3S5M5ZIE.1 size=1034809 2024-03-04T17:50:35.103 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.105+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=PCZW3S5M5ZIE tag=ABGR0AV969CH type=write size=3131961 2024-03-04T17:50:35.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.109+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=2HS7A69J25WW type=write size=719262 2024-03-04T17:50:35.111 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.113+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=FW2B1ZOQXWX8 type=del 2024-03-04T17:50:35.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.117+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~ME9IE3GEZHE9.0 size=1048576 2024-03-04T17:50:35.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.121+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~ME9IE3GEZHE9.1 size=49352 2024-03-04T17:50:35.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.121+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=ME9IE3GEZHE9 tag=IGDFSK1Q1T8P type=write size=2146504 2024-03-04T17:50:35.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.125+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=Q3H46HA75IEC type=del 2024-03-04T17:50:35.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.133+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~0Z23ETJZX2TY.0 size=1048576 2024-03-04T17:50:35.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.133+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~0Z23ETJZX2TY.1 size=1048576 2024-03-04T17:50:35.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.137+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~0Z23ETJZX2TY.2 size=323112 2024-03-04T17:50:35.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.137+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=0Z23ETJZX2TY tag=GOZBDAPWRWUR type=write size=3468840 2024-03-04T17:50:35.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.141+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~8OTW3RL1QZSE.0 size=244160 2024-03-04T17:50:35.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.149+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=8OTW3RL1QZSE tag=NULB46FGG1XL type=write size=1292736 2024-03-04T17:50:35.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.157+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~GZRQE3BF40AF.0 size=1048576 2024-03-04T17:50:35.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.157+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~GZRQE3BF40AF.1 size=1048576 2024-03-04T17:50:35.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.161+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~GZRQE3BF40AF.2 size=235768 2024-03-04T17:50:35.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.161+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=GZRQE3BF40AF tag=3IIW1XAG9307 type=write size=3381496 2024-03-04T17:50:35.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.165+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=DN81X2M6ANSY type=del 2024-03-04T17:50:35.170 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.173+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=4Z2OZPIFQOEQ type=del size=2463840 2024-03-04T17:50:35.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.177+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=CIUUXXYH64LO type=del 2024-03-04T17:50:35.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.181+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=LNI7O5LI10DZ type=del size=867046 2024-03-04T17:50:35.183 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.185+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=ARBU8RRU0G36 type=del 2024-03-04T17:50:35.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.189+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=R7GARSSLXW28 type=del size=1673373 2024-03-04T17:50:35.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.197+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=RAQKAEPN59YA type=write 2024-03-04T17:50:35.199 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.201+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=GK2N1NEYLANQ type=del size=1149081 2024-03-04T17:50:35.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.205+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~W1UWVBCUJJ60.0 size=646990 2024-03-04T17:50:35.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.209+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=W1UWVBCUJJ60 tag=IRVLB08K78TK type=write size=1695566 2024-03-04T17:50:35.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.217+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~2KTEQ81YJ6EW.0 size=1048576 2024-03-04T17:50:35.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.221+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~2KTEQ81YJ6EW.1 size=1048576 2024-03-04T17:50:35.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.221+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~2KTEQ81YJ6EW.2 size=189428 2024-03-04T17:50:35.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.221+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=2KTEQ81YJ6EW tag=39F33KAJZ9RO type=write size=3335156 2024-03-04T17:50:35.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.229+0000 7fda08af98c0 1 < suggested remove operation key=IJSYAVR5SWPS tag=OK2JA2L09PO1 type=del 2024-03-04T17:50:35.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.233+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=TIX13RVHFCQD type=write size=83302 2024-03-04T17:50:35.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.241+0000 7fda08af98c0 1 < suggested update operation key=VJVORUG95FLC tag=IJR5VQXV0FGR type=del size=1884190 2024-03-04T17:50:35.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.245+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=MGTJ4UYURVVN type=del size=821720 2024-03-04T17:50:35.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.249+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~PQZ8M42T2Y6K.0 size=1048576 2024-03-04T17:50:35.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.249+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~PQZ8M42T2Y6K.1 size=1048576 2024-03-04T17:50:35.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.253+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~PQZ8M42T2Y6K.2 size=195757 2024-03-04T17:50:35.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.261+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=PQZ8M42T2Y6K tag=Q2MVNT19XVR5 type=write size=3341485 2024-03-04T17:50:35.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.261+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=8C06MITG150R type=del size=3857386 2024-03-04T17:50:35.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.265+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~CHR6UOC7XJZ8.0 size=1048576 2024-03-04T17:50:35.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.269+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~CHR6UOC7XJZ8.1 size=1048576 2024-03-04T17:50:35.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.273+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~CHR6UOC7XJZ8.2 size=227344 2024-03-04T17:50:35.277 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.277+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=CHR6UOC7XJZ8 tag=V3WL89P84Q54 type=write size=3373072 2024-03-04T17:50:35.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.285+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~S856DIQC2AP0.0 size=1048576 2024-03-04T17:50:35.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.289+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~S856DIQC2AP0.1 size=133503 2024-03-04T17:50:35.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.289+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=S856DIQC2AP0 tag=RN9SHTQ4XCU9 type=write size=2230655 2024-03-04T17:50:35.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.293+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~ABY381LQW3AA.0 size=1048576 2024-03-04T17:50:35.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.297+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~ABY381LQW3AA.1 size=201782 2024-03-04T17:50:35.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.305+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=ABY381LQW3AA tag=8N6G4KJ9P6TQ type=write size=2298934 2024-03-04T17:50:35.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.309+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=7CPBI1YYE989 type=write size=116681 2024-03-04T17:50:35.310 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.313+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~2O0ASKH49ZXQ.0 size=395872 2024-03-04T17:50:35.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.317+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=2O0ASKH49ZXQ tag=8YLHQJX0UHOC type=write size=1444448 2024-03-04T17:50:35.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.321+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=RILYEHVPJHK6 type=del size=3026244 2024-03-04T17:50:35.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.325+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=O4GFPHSUGZ01 type=del 2024-03-04T17:50:35.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.329+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=YE5M13J6JACJ type=write size=971199 2024-03-04T17:50:35.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.333+0000 7fda08af98c0 1 < canceled operation key=1OZKPHMPV3OQ tag=2HS7A69J25WW type=write size=719262 2024-03-04T17:50:35.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.337+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=3LGPG1RGOAXT type=del size=3658464 2024-03-04T17:50:35.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.341+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~ME9IE3GEZHE9.0 size=1048576 2024-03-04T17:50:35.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.341+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~ME9IE3GEZHE9.1 size=49352 2024-03-04T17:50:35.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.349+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=ME9IE3GEZHE9 tag=IGDFSK1Q1T8P type=write size=2146504 2024-03-04T17:50:35.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.353+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=3H8K2NTAIFAA type=del size=707309 2024-03-04T17:50:35.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.357+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~0Z23ETJZX2TY.0 size=1048576 2024-03-04T17:50:35.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.361+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~0Z23ETJZX2TY.1 size=1048576 2024-03-04T17:50:35.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.361+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~0Z23ETJZX2TY.2 size=323112 2024-03-04T17:50:35.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.369+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=0Z23ETJZX2TY tag=GOZBDAPWRWUR type=write size=3468840 2024-03-04T17:50:35.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.373+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=EP67M6WDLY4Y type=del size=4007494 2024-03-04T17:50:35.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.373+0000 7fda08af98c0 1 < canceled part key=_multipart_8YL7C9PI58TK.2~GZRQE3BF40AF.0 size=1048576 2024-03-04T17:50:35.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.373+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~GZRQE3BF40AF.1 size=1048576 2024-03-04T17:50:35.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.377+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~GZRQE3BF40AF.2 size=235768 2024-03-04T17:50:35.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.381+0000 7fda08af98c0 1 < canceled multipart upload key=8YL7C9PI58TK upload=GZRQE3BF40AF tag=3IIW1XAG9307 type=write size=3381496 2024-03-04T17:50:35.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.385+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=8Z8P2VBU3LCJ type=write size=660333 2024-03-04T17:50:35.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.389+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=4Z2OZPIFQOEQ type=del 2024-03-04T17:50:35.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.393+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=K94F1LMPN9L3 type=del size=1259249 2024-03-04T17:50:35.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.397+0000 7fda08af98c0 -1 < failed to complete operation key=IJSYAVR5SWPS tag=LNI7O5LI10DZ type=del size=867046: (22) Invalid argument 2024-03-04T17:50:35.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.405+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~02S3WS2NGJ34.0 size=1048576 2024-03-04T17:50:35.404 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.405+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~02S3WS2NGJ34.1 size=1048576 2024-03-04T17:50:35.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.409+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~02S3WS2NGJ34.2 size=789642 2024-03-04T17:50:35.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.409+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=02S3WS2NGJ34 tag=ZRIPXFS7WWSO type=write size=3935370 2024-03-04T17:50:35.410 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.413+0000 7fda08af98c0 -1 < failed to complete operation key=M5BMX7B7RZYL tag=R7GARSSLXW28 type=del size=1673373: (22) Invalid argument 2024-03-04T17:50:35.414 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.417+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=8EWEGOQA9GQJ type=del size=1652529 2024-03-04T17:50:35.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.421+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=GK2N1NEYLANQ type=del 2024-03-04T17:50:35.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.425+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~D1OF1MKQCLUA.0 size=96730 2024-03-04T17:50:35.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.425+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=D1OF1MKQCLUA tag=PRMDHY5P5VD8 type=write size=1145306 2024-03-04T17:50:35.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.429+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~2KTEQ81YJ6EW.0 size=1048576 2024-03-04T17:50:35.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.433+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~2KTEQ81YJ6EW.1 size=1048576 2024-03-04T17:50:35.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.437+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~2KTEQ81YJ6EW.2 size=189428 2024-03-04T17:50:35.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.441+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=2KTEQ81YJ6EW tag=39F33KAJZ9RO type=write size=3335156 2024-03-04T17:50:35.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.449+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~S28BGXOCMC62.0 size=1048576 2024-03-04T17:50:35.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.453+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~S28BGXOCMC62.1 size=430992 2024-03-04T17:50:35.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.453+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=S28BGXOCMC62 tag=J3XUP2SR2QYV type=write size=2528144 2024-03-04T17:50:35.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.457+0000 7fda08af98c0 1 < completed write operation key=AO3QS6CM7I5D tag=TIX13RVHFCQD type=write size=83302 2024-03-04T17:50:35.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.457+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=GUJ1UIN4VWZV type=del size=1034658 2024-03-04T17:50:35.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.461+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=MGTJ4UYURVVN type=del 2024-03-04T17:50:35.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.469+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~7AV69WMA4MRZ.0 size=817861 2024-03-04T17:50:35.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.469+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=7AV69WMA4MRZ tag=M9BTUEUAUHYP type=write size=1866437 2024-03-04T17:50:35.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.473+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=8C06MITG150R type=del 2024-03-04T17:50:35.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.477+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=APRE0WCR5Q1X type=del size=1399387 2024-03-04T17:50:35.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.481+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~S856DIQC2AP0.0 size=1048576 2024-03-04T17:50:35.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.485+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~S856DIQC2AP0.1 size=133503 2024-03-04T17:50:35.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.489+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=S856DIQC2AP0 tag=RN9SHTQ4XCU9 type=write size=2230655 2024-03-04T17:50:35.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.493+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=4854WWX29CTY type=del size=2332809 2024-03-04T17:50:35.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.497+0000 7fda08af98c0 1 < completed write operation key=JA4JK99I0942 tag=7CPBI1YYE989 type=write size=116681 2024-03-04T17:50:35.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.501+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=SAIA1M7NYRXY type=write size=317231 2024-03-04T17:50:35.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.505+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=RILYEHVPJHK6 type=del 2024-03-04T17:50:35.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.509+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=060Z8K3S1KYP type=del size=3837010 2024-03-04T17:50:35.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.513+0000 7fda08af98c0 1 < suggested remove operation key=DJCCQ6S0E7UG tag=YE5M13J6JACJ type=write 2024-03-04T17:50:35.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.521+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~CP1YILGL4J9S.0 size=833094 2024-03-04T17:50:35.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.521+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=CP1YILGL4J9S tag=GP71P47X3841 type=write size=1881670 2024-03-04T17:50:35.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.525+0000 7fda08af98c0 -1 < failed to complete operation key=DJCCQ6S0E7UG tag=3LGPG1RGOAXT type=del size=3658464: (22) Invalid argument 2024-03-04T17:50:35.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.529+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=BF12PS63PT9L type=del size=2423985 2024-03-04T17:50:35.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.537+0000 7fda08af98c0 1 < suggested update operation key=IJSYAVR5SWPS tag=3H8K2NTAIFAA type=del size=707309 2024-03-04T17:50:35.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.541+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=3D6OCRTDKP57 type=del size=2577664 2024-03-04T17:50:35.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.545+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=EP67M6WDLY4Y type=del 2024-03-04T17:50:35.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.549+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=Z8X1HNEQ9L2K type=del size=624967 2024-03-04T17:50:35.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.553+0000 7fda08af98c0 1 < completed write operation key=X5MEJ7G8X9HE tag=8Z8P2VBU3LCJ type=write size=660333 2024-03-04T17:50:35.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.557+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=O0T4JLO5ZEMS type=write size=928884 2024-03-04T17:50:35.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.561+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=K94F1LMPN9L3 type=del 2024-03-04T17:50:35.565 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.565+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~GEATBGUZ7YL6.0 size=185780 2024-03-04T17:50:35.565 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.565+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=GEATBGUZ7YL6 tag=4UR5A6TWRGT1 type=write size=1234356 2024-03-04T17:50:35.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.569+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~02S3WS2NGJ34.0 size=1048576 2024-03-04T17:50:35.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.573+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~02S3WS2NGJ34.1 size=1048576 2024-03-04T17:50:35.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.573+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~02S3WS2NGJ34.2 size=789642 2024-03-04T17:50:35.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.581+0000 7fda08af98c0 1 < canceled multipart upload key=JA4JK99I0942 upload=02S3WS2NGJ34 tag=ZRIPXFS7WWSO type=write size=3935370 2024-03-04T17:50:35.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.589+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~P8214FTIR4B0.0 size=1048576 2024-03-04T17:50:35.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.593+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~P8214FTIR4B0.1 size=595924 2024-03-04T17:50:35.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.593+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=P8214FTIR4B0 tag=VSVSKNN2RH9D type=write size=2693076 2024-03-04T17:50:35.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.597+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=8EWEGOQA9GQJ type=del 2024-03-04T17:50:35.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.601+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=DXH3D5FNQS30 type=del size=2928771 2024-03-04T17:50:35.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.605+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~D1OF1MKQCLUA.0 size=96730 2024-03-04T17:50:35.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.609+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=D1OF1MKQCLUA tag=PRMDHY5P5VD8 type=write size=1145306 2024-03-04T17:50:35.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.613+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=YWO1NHA9SUXX type=write size=915686 2024-03-04T17:50:35.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.613+0000 7fda08af98c0 1 < canceled part key=_multipart_FMPZ2548LBBF.2~S28BGXOCMC62.0 size=1048576 2024-03-04T17:50:35.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.613+0000 7fda08af98c0 1 < canceled part key=_multipart_FMPZ2548LBBF.2~S28BGXOCMC62.1 size=430992 2024-03-04T17:50:35.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.621+0000 7fda08af98c0 1 < canceled multipart upload key=FMPZ2548LBBF upload=S28BGXOCMC62 tag=J3XUP2SR2QYV type=write size=2528144 2024-03-04T17:50:35.627 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.625+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~BIR4URZSQXAT.0 size=1048576 2024-03-04T17:50:35.628 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.629+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~BIR4URZSQXAT.1 size=1048576 2024-03-04T17:50:35.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.637+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~BIR4URZSQXAT.2 size=258732 2024-03-04T17:50:35.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.637+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=BIR4URZSQXAT tag=K3AF48ZC50OX type=write size=3404460 2024-03-04T17:50:35.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.641+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=GUJ1UIN4VWZV type=del 2024-03-04T17:50:35.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.641+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=3JMGYJQE3J2C type=write size=271221 2024-03-04T17:50:35.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.645+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~7AV69WMA4MRZ.0 size=817861 2024-03-04T17:50:35.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.653+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=7AV69WMA4MRZ tag=M9BTUEUAUHYP type=write size=1866437 2024-03-04T17:50:35.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.657+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=GXKITB08L8HQ type=del size=1007788 2024-03-04T17:50:35.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.657+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=APRE0WCR5Q1X type=del 2024-03-04T17:50:35.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.665+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~O8T9PNKY69RT.0 size=196815 2024-03-04T17:50:35.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.665+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=O8T9PNKY69RT tag=YL2HGDRP7BJ2 type=write size=1245391 2024-03-04T17:50:35.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.665+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=4854WWX29CTY type=del 2024-03-04T17:50:35.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.673+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~0EBWQI4GIC1R.0 size=1048576 2024-03-04T17:50:35.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.673+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~0EBWQI4GIC1R.1 size=1048576 2024-03-04T17:50:35.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.677+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~0EBWQI4GIC1R.2 size=631902 2024-03-04T17:50:35.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.677+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=0EBWQI4GIC1R tag=XL0HPSDLENGN type=write size=3777630 2024-03-04T17:50:35.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.681+0000 7fda08af98c0 1 < completed write operation key=HCA7QLGM2O4O tag=SAIA1M7NYRXY type=write size=317231 2024-03-04T17:50:35.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.681+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=47PDK7MVZEKB type=del size=2109372 2024-03-04T17:50:35.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.685+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=060Z8K3S1KYP type=del 2024-03-04T17:50:35.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.689+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=ZAMG42JU86EZ type=del size=2002938 2024-03-04T17:50:35.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.693+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~CP1YILGL4J9S.0 size=833094 2024-03-04T17:50:35.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.697+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=CP1YILGL4J9S tag=GP71P47X3841 type=write size=1881670 2024-03-04T17:50:35.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.701+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=WWTG763SQ32W type=del size=81193 2024-03-04T17:50:35.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.705+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=BF12PS63PT9L type=del 2024-03-04T17:50:35.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.709+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~AMBGPGPK8EXW.0 size=821638 2024-03-04T17:50:35.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.709+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=AMBGPGPK8EXW tag=WWKA6YONH7ZG type=write size=1870214 2024-03-04T17:50:35.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.713+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=3D6OCRTDKP57 type=del 2024-03-04T17:50:35.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.721+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~HJER5O6YLSXO.0 size=1048576 2024-03-04T17:50:35.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.721+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~HJER5O6YLSXO.1 size=1048576 2024-03-04T17:50:35.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.725+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~HJER5O6YLSXO.2 size=729833 2024-03-04T17:50:35.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.725+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=HJER5O6YLSXO tag=POEPTCEG2848 type=write size=3875561 2024-03-04T17:50:35.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.729+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=Z8X1HNEQ9L2K type=del 2024-03-04T17:50:35.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.733+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=X22ESS1S3XRQ type=del size=1330576 2024-03-04T17:50:35.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.737+0000 7fda08af98c0 1 < completed write operation key=BYQFZSJ3WH07 tag=O0T4JLO5ZEMS type=write size=928884 2024-03-04T17:50:35.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.741+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~NV7WIQX7P11A.0 size=1048576 2024-03-04T17:50:35.742 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.745+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~NV7WIQX7P11A.1 size=954531 2024-03-04T17:50:35.742 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.745+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=NV7WIQX7P11A tag=DV1XH9GMUBQR type=write size=3051683 2024-03-04T17:50:35.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.749+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~GEATBGUZ7YL6.0 size=185780 2024-03-04T17:50:35.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.753+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=GEATBGUZ7YL6 tag=4UR5A6TWRGT1 type=write size=1234356 2024-03-04T17:50:35.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.761+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~OBLX7HEEE1AB.0 size=43350 2024-03-04T17:50:35.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.761+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=OBLX7HEEE1AB tag=MBM5NDW8AL02 type=write size=1091926 2024-03-04T17:50:35.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.761+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~P8214FTIR4B0.0 size=1048576 2024-03-04T17:50:35.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.765+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~P8214FTIR4B0.1 size=595924 2024-03-04T17:50:35.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.769+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=P8214FTIR4B0 tag=VSVSKNN2RH9D type=write size=2693076 2024-03-04T17:50:35.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.773+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=AJRC1UEVJCKZ type=write size=203386 2024-03-04T17:50:35.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.777+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=DXH3D5FNQS30 type=del 2024-03-04T17:50:35.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.781+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=Q32RM0K0X80L type=del size=252754 2024-03-04T17:50:35.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.785+0000 7fda08af98c0 1 < completed write operation key=TOFWDFH0HL3Q tag=YWO1NHA9SUXX type=write size=915686 2024-03-04T17:50:35.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.789+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=GJUA251RL76D type=write size=151637 2024-03-04T17:50:35.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.793+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~BIR4URZSQXAT.0 size=1048576 2024-03-04T17:50:35.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.797+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~BIR4URZSQXAT.1 size=1048576 2024-03-04T17:50:35.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.797+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~BIR4URZSQXAT.2 size=258732 2024-03-04T17:50:35.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.805+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=BIR4URZSQXAT tag=K3AF48ZC50OX type=write size=3404460 2024-03-04T17:50:35.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.809+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~39E38GIWK0OR.0 size=1048576 2024-03-04T17:50:35.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.813+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~39E38GIWK0OR.1 size=620941 2024-03-04T17:50:35.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.813+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=39E38GIWK0OR tag=1RQ1PIOARFBA type=write size=2718093 2024-03-04T17:50:35.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.817+0000 7fda08af98c0 1 < completed write operation key=IJSYAVR5SWPS tag=3JMGYJQE3J2C type=write size=271221 2024-03-04T17:50:35.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.825+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~NNHVUZNSINHN.0 size=1048576 2024-03-04T17:50:35.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.825+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~NNHVUZNSINHN.1 size=1048576 2024-03-04T17:50:35.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.829+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~NNHVUZNSINHN.2 size=507817 2024-03-04T17:50:35.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.829+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=NNHVUZNSINHN tag=14VBJ1ZXSXD6 type=write size=3653545 2024-03-04T17:50:35.831 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.833+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=GXKITB08L8HQ type=del 2024-03-04T17:50:35.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.837+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=PRIOUCUR3783 type=del size=1542476 2024-03-04T17:50:35.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.841+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~O8T9PNKY69RT.0 size=196815 2024-03-04T17:50:35.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.845+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=O8T9PNKY69RT tag=YL2HGDRP7BJ2 type=write size=1245391 2024-03-04T17:50:35.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.849+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=74O921WT8C4T type=write size=728294 2024-03-04T17:50:35.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.853+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~0EBWQI4GIC1R.0 size=1048576 2024-03-04T17:50:35.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.853+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~0EBWQI4GIC1R.1 size=1048576 2024-03-04T17:50:35.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.857+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~0EBWQI4GIC1R.2 size=631902 2024-03-04T17:50:35.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.861+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=0EBWQI4GIC1R tag=XL0HPSDLENGN type=write size=3777630 2024-03-04T17:50:35.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.869+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~4S2GZD1611WI.0 size=1048576 2024-03-04T17:50:35.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.873+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~4S2GZD1611WI.1 size=115343 2024-03-04T17:50:35.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.873+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=4S2GZD1611WI tag=NAQUB4T6H0NE type=write size=2212495 2024-03-04T17:50:35.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.877+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=47PDK7MVZEKB type=del 2024-03-04T17:50:35.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.877+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=GVTHEEFI4OV5 type=del size=2247187 2024-03-04T17:50:35.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.885+0000 7fda08af98c0 1 < suggested update operation key=TOFWDFH0HL3Q tag=ZAMG42JU86EZ type=del size=2002938 2024-03-04T17:50:35.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.893+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~5PZ2XF76EJ27.0 size=868115 2024-03-04T17:50:35.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.893+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=5PZ2XF76EJ27 tag=RVP9NDX8AZMW type=write size=1916691 2024-03-04T17:50:35.894 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.897+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=WWTG763SQ32W type=del 2024-03-04T17:50:35.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.901+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=SGMBFWYUNKHO type=del size=3816118 2024-03-04T17:50:35.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.905+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~AMBGPGPK8EXW.0 size=821638 2024-03-04T17:50:35.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.909+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=AMBGPGPK8EXW tag=WWKA6YONH7ZG type=write size=1870214 2024-03-04T17:50:35.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.913+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=786PMM9CJP4B type=write size=889101 2024-03-04T17:50:35.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.917+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~HJER5O6YLSXO.0 size=1048576 2024-03-04T17:50:35.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.917+0000 7fda08af98c0 1 < canceled part key=_multipart_HCA7QLGM2O4O.2~HJER5O6YLSXO.1 size=1048576 2024-03-04T17:50:35.919 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.921+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~HJER5O6YLSXO.2 size=729833 2024-03-04T17:50:35.925 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.925+0000 7fda08af98c0 1 < canceled multipart upload key=HCA7QLGM2O4O upload=HJER5O6YLSXO tag=POEPTCEG2848 type=write size=3875561 2024-03-04T17:50:35.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.929+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=FE32SISJE780 type=del size=2711792 2024-03-04T17:50:35.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.937+0000 7fda08af98c0 1 < suggested update operation key=8YL7C9PI58TK tag=X22ESS1S3XRQ type=del size=1330576 2024-03-04T17:50:35.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.945+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=SERWVOJ6YQDE type=write size=635955 2024-03-04T17:50:35.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.949+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~NV7WIQX7P11A.0 size=1048576 2024-03-04T17:50:35.949 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.949+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~NV7WIQX7P11A.1 size=954531 2024-03-04T17:50:35.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.957+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=NV7WIQX7P11A tag=DV1XH9GMUBQR type=write size=3051683 2024-03-04T17:50:35.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.961+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=AJTK8PA7PARH type=del size=4041667 2024-03-04T17:50:35.964 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.965+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~OBLX7HEEE1AB.0 size=43350 2024-03-04T17:50:35.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.973+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=OBLX7HEEE1AB tag=MBM5NDW8AL02 type=write size=1091926 2024-03-04T17:50:35.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.973+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=QJWY8WUR7GJ5 type=del size=327664 2024-03-04T17:50:35.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.977+0000 7fda08af98c0 1 < completed write operation key=DJCCQ6S0E7UG tag=AJRC1UEVJCKZ type=write size=203386 2024-03-04T17:50:35.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.985+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~GSAGTY2N4M8L.0 size=1048576 2024-03-04T17:50:35.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.989+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~GSAGTY2N4M8L.1 size=1048576 2024-03-04T17:50:35.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.989+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~GSAGTY2N4M8L.2 size=234468 2024-03-04T17:50:35.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.989+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=GSAGTY2N4M8L tag=T4TT89WVQO8B type=write size=3380196 2024-03-04T17:50:35.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.993+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=Q32RM0K0X80L type=del 2024-03-04T17:50:35.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:35.997+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=2LECD1IGIZG9 type=del size=3013789 2024-03-04T17:50:35.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.001+0000 7fda08af98c0 -1 < failed to complete operation key=8YL7C9PI58TK tag=GJUA251RL76D type=write size=151637: (22) Invalid argument 2024-03-04T17:50:36.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.009+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~R15VW5L8JMC7.0 size=1048576 2024-03-04T17:50:36.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.009+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~R15VW5L8JMC7.1 size=366585 2024-03-04T17:50:36.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.009+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=R15VW5L8JMC7 tag=5BGRMVWD6G7D type=write size=2463737 2024-03-04T17:50:36.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.013+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~39E38GIWK0OR.0 size=1048576 2024-03-04T17:50:36.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.017+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~39E38GIWK0OR.1 size=620941 2024-03-04T17:50:36.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.025+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=39E38GIWK0OR tag=1RQ1PIOARFBA type=write size=2718093 2024-03-04T17:50:36.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.029+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=X8RVNZAANLFR type=del size=2891294 2024-03-04T17:50:36.027 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.029+0000 7fda08af98c0 1 < canceled part key=_multipart_0IGSMFORCSL9.2~NNHVUZNSINHN.0 size=1048576 2024-03-04T17:50:36.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.033+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~NNHVUZNSINHN.1 size=1048576 2024-03-04T17:50:36.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.037+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~NNHVUZNSINHN.2 size=507817 2024-03-04T17:50:36.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.041+0000 7fda08af98c0 1 < canceled multipart upload key=0IGSMFORCSL9 upload=NNHVUZNSINHN tag=14VBJ1ZXSXD6 type=write size=3653545 2024-03-04T17:50:36.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.045+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=EM1EIWZ157WI type=write size=801353 2024-03-04T17:50:36.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.049+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=PRIOUCUR3783 type=del 2024-03-04T17:50:36.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.057+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~WC6VW1WJMQRD.0 size=207340 2024-03-04T17:50:36.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.057+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=WC6VW1WJMQRD tag=I9NMW2C3BYMS type=write size=1255916 2024-03-04T17:50:36.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.061+0000 7fda08af98c0 1 < completed write operation key=XG5I3SDRE5JM tag=74O921WT8C4T type=write size=728294 2024-03-04T17:50:36.067 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.069+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~PU23YS18T171.0 size=1048576 2024-03-04T17:50:36.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.073+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~PU23YS18T171.1 size=407125 2024-03-04T17:50:36.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.073+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=PU23YS18T171 tag=HYIJRV99231A type=write size=2504277 2024-03-04T17:50:36.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.077+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~4S2GZD1611WI.0 size=1048576 2024-03-04T17:50:36.077 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.077+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~4S2GZD1611WI.1 size=115343 2024-03-04T17:50:36.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.085+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=4S2GZD1611WI tag=NAQUB4T6H0NE type=write size=2212495 2024-03-04T17:50:36.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.093+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~WPIDZSY9TXV9.0 size=1048576 2024-03-04T17:50:36.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.093+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~WPIDZSY9TXV9.1 size=25940 2024-03-04T17:50:36.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.093+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=WPIDZSY9TXV9 tag=LV6GNBJE1E3Q type=write size=2123092 2024-03-04T17:50:36.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.097+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=GVTHEEFI4OV5 type=del 2024-03-04T17:50:36.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.101+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=K1MSNX0EAHRE type=write size=893666 2024-03-04T17:50:36.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.101+0000 7fda08af98c0 1 < canceled part key=_multipart_XG5I3SDRE5JM.2~5PZ2XF76EJ27.0 size=868115 2024-03-04T17:50:36.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.109+0000 7fda08af98c0 1 < canceled multipart upload key=XG5I3SDRE5JM upload=5PZ2XF76EJ27 tag=RVP9NDX8AZMW type=write size=1916691 2024-03-04T17:50:36.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.117+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~OHJ37I646ZXD.0 size=625877 2024-03-04T17:50:36.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.117+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=OHJ37I646ZXD tag=6DUU249SS23F type=write size=1674453 2024-03-04T17:50:36.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.121+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=SGMBFWYUNKHO type=del 2024-03-04T17:50:36.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.125+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=F784JN7HVCB7 type=write size=994112 2024-03-04T17:50:36.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.129+0000 7fda08af98c0 1 < completed write operation key=TOFWDFH0HL3Q tag=786PMM9CJP4B type=write size=889101 2024-03-04T17:50:36.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.133+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=2545VDMPEUB8 type=del size=219981 2024-03-04T17:50:36.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.137+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=FE32SISJE780 type=del 2024-03-04T17:50:36.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.141+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=EBMXD8EWWCY0 type=del size=1195041 2024-03-04T17:50:36.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.149+0000 7fda08af98c0 1 < suggested remove operation key=09KBCA2XURTF tag=SERWVOJ6YQDE type=write 2024-03-04T17:50:36.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.149+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=O31NYLDET5LF type=del size=1494648 2024-03-04T17:50:36.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.153+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=AJTK8PA7PARH type=del 2024-03-04T17:50:36.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.161+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~6S68YTXN86TQ.0 size=1048576 2024-03-04T17:50:36.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.165+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~6S68YTXN86TQ.1 size=720963 2024-03-04T17:50:36.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.165+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=6S68YTXN86TQ tag=3CP7DYDU6U0Z type=write size=2818115 2024-03-04T17:50:36.168 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.169+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=QJWY8WUR7GJ5 type=del 2024-03-04T17:50:36.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.173+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=CLUTOIP1G0GM type=del size=3807911 2024-03-04T17:50:36.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.177+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~GSAGTY2N4M8L.0 size=1048576 2024-03-04T17:50:36.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.177+0000 7fda08af98c0 1 < canceled part key=_multipart_XG2XYBCMUNY5.2~GSAGTY2N4M8L.1 size=1048576 2024-03-04T17:50:36.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.177+0000 7fda08af98c0 1 < canceled part key=_multipart_XG2XYBCMUNY5.2~GSAGTY2N4M8L.2 size=234468 2024-03-04T17:50:36.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.185+0000 7fda08af98c0 1 < canceled multipart upload key=XG2XYBCMUNY5 upload=GSAGTY2N4M8L tag=T4TT89WVQO8B type=write size=3380196 2024-03-04T17:50:36.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.189+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=WO1N3NJBR2ZJ type=del size=176269 2024-03-04T17:50:36.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.193+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=2LECD1IGIZG9 type=del 2024-03-04T17:50:36.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.197+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=EH0BCSWQ5FPF type=del size=387660 2024-03-04T17:50:36.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.201+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~R15VW5L8JMC7.0 size=1048576 2024-03-04T17:50:36.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.201+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~R15VW5L8JMC7.1 size=366585 2024-03-04T17:50:36.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.209+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=R15VW5L8JMC7 tag=5BGRMVWD6G7D type=write size=2463737 2024-03-04T17:50:36.214 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.217+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~82SVBIC0GPW0.0 size=1048576 2024-03-04T17:50:36.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.217+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~82SVBIC0GPW0.1 size=179826 2024-03-04T17:50:36.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.217+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=82SVBIC0GPW0 tag=JFXFGNW1F9JX type=write size=2276978 2024-03-04T17:50:36.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.225+0000 7fda08af98c0 1 < suggested remove operation key=C4B45GL40IMZ tag=X8RVNZAANLFR type=del 2024-03-04T17:50:36.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.233+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~41QDRL3LPE2G.0 size=1048576 2024-03-04T17:50:36.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.233+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~41QDRL3LPE2G.1 size=211454 2024-03-04T17:50:36.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.233+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=41QDRL3LPE2G tag=TABBVA1JG6C1 type=write size=2308606 2024-03-04T17:50:36.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.237+0000 7fda08af98c0 1 < completed write operation key=G0BHMSCO0XLF tag=EM1EIWZ157WI type=write size=801353 2024-03-04T17:50:36.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.245+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~6QF5EBXAAOAM.0 size=1048576 2024-03-04T17:50:36.245 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.245+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~6QF5EBXAAOAM.1 size=451550 2024-03-04T17:50:36.245 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.245+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=6QF5EBXAAOAM tag=NFR0FNPAMT5G type=write size=2548702 2024-03-04T17:50:36.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.249+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~WC6VW1WJMQRD.0 size=207340 2024-03-04T17:50:36.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.257+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=WC6VW1WJMQRD tag=I9NMW2C3BYMS type=write size=1255916 2024-03-04T17:50:36.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.261+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=8T18IBIK5FWH type=del size=2949816 2024-03-04T17:50:36.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.261+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~PU23YS18T171.0 size=1048576 2024-03-04T17:50:36.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.265+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~PU23YS18T171.1 size=407125 2024-03-04T17:50:36.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.269+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=PU23YS18T171 tag=HYIJRV99231A type=write size=2504277 2024-03-04T17:50:36.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.273+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=VJCU3OS8VMVC type=del size=2485292 2024-03-04T17:50:36.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.277+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~WPIDZSY9TXV9.0 size=1048576 2024-03-04T17:50:36.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.281+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~WPIDZSY9TXV9.1 size=25940 2024-03-04T17:50:36.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.285+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=WPIDZSY9TXV9 tag=LV6GNBJE1E3Q type=write size=2123092 2024-03-04T17:50:36.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.293+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~TJZSHAOZQ4MU.0 size=944760 2024-03-04T17:50:36.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.293+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=TJZSHAOZQ4MU tag=SPG169RPOGMD type=write size=1993336 2024-03-04T17:50:36.299 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.301+0000 7fda08af98c0 1 < completed write operation key=HCA7QLGM2O4O tag=K1MSNX0EAHRE type=write size=893666 2024-03-04T17:50:36.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.305+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=9MUFJ0QDZQ40 type=del size=2827087 2024-03-04T17:50:36.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.309+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~OHJ37I646ZXD.0 size=625877 2024-03-04T17:50:36.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.317+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=OHJ37I646ZXD tag=6DUU249SS23F type=write size=1674453 2024-03-04T17:50:36.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.317+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=KZK5WBWQLBQX type=write size=665383 2024-03-04T17:50:36.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.321+0000 7fda08af98c0 1 < canceled operation key=GDGTJZ21TG32 tag=F784JN7HVCB7 type=write size=994112 2024-03-04T17:50:36.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.329+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~NRSBOSD3JRYY.0 size=1048576 2024-03-04T17:50:36.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.333+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~NRSBOSD3JRYY.1 size=1048576 2024-03-04T17:50:36.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.337+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~NRSBOSD3JRYY.2 size=883934 2024-03-04T17:50:36.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.337+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=NRSBOSD3JRYY tag=OJTTKUBYQR7U type=write size=4029662 2024-03-04T17:50:36.340 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.341+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=2545VDMPEUB8 type=del 2024-03-04T17:50:36.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.345+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=2YFLS70KBP9W type=del size=3835129 2024-03-04T17:50:36.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.349+0000 7fda08af98c0 -1 < failed to complete operation key=C4B45GL40IMZ tag=EBMXD8EWWCY0 type=del size=1195041: (22) Invalid argument 2024-03-04T17:50:36.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.353+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=AXKL7L0SISYL type=del size=514744 2024-03-04T17:50:36.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.357+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=O31NYLDET5LF type=del 2024-03-04T17:50:36.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.365+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~QHA7GOXM4P59.0 size=39886 2024-03-04T17:50:36.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.365+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=QHA7GOXM4P59 tag=5679IM655MYA type=write size=1088462 2024-03-04T17:50:36.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.365+0000 7fda08af98c0 1 < canceled part key=_multipart_FMPZ2548LBBF.2~6S68YTXN86TQ.0 size=1048576 2024-03-04T17:50:36.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.369+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~6S68YTXN86TQ.1 size=720963 2024-03-04T17:50:36.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.373+0000 7fda08af98c0 1 < canceled multipart upload key=FMPZ2548LBBF upload=6S68YTXN86TQ tag=3CP7DYDU6U0Z type=write size=2818115 2024-03-04T17:50:36.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.377+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=393PL2K1VDZI type=del size=2068157 2024-03-04T17:50:36.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.385+0000 7fda08af98c0 1 < suggested update operation key=3UWSL7NJLACY tag=CLUTOIP1G0GM type=del size=3807911 2024-03-04T17:50:36.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.389+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=QOQMA48AC7Q5 type=del size=2858908 2024-03-04T17:50:36.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.393+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=WO1N3NJBR2ZJ type=del 2024-03-04T17:50:36.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.397+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~YEDGRGIFUM5N.0 size=1048576 2024-03-04T17:50:36.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.401+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~YEDGRGIFUM5N.1 size=1048576 2024-03-04T17:50:36.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.405+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~YEDGRGIFUM5N.2 size=58783 2024-03-04T17:50:36.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.405+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=YEDGRGIFUM5N tag=RE89QWI01APC type=write size=3204511 2024-03-04T17:50:36.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.409+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=EH0BCSWQ5FPF type=del 2024-03-04T17:50:36.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.413+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=MFYZA5ZMV8EK type=write size=937248 2024-03-04T17:50:36.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.417+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~82SVBIC0GPW0.0 size=1048576 2024-03-04T17:50:36.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.421+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~82SVBIC0GPW0.1 size=179826 2024-03-04T17:50:36.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.425+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=82SVBIC0GPW0 tag=JFXFGNW1F9JX type=write size=2276978 2024-03-04T17:50:36.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.429+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=B0QSJFSHOKIA type=del size=2565758 2024-03-04T17:50:36.432 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.433+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~41QDRL3LPE2G.0 size=1048576 2024-03-04T17:50:36.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.437+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~41QDRL3LPE2G.1 size=211454 2024-03-04T17:50:36.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.441+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=41QDRL3LPE2G tag=TABBVA1JG6C1 type=write size=2308606 2024-03-04T17:50:36.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.445+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=110HWH66PYK5 type=del size=997415 2024-03-04T17:50:36.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.449+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~6QF5EBXAAOAM.0 size=1048576 2024-03-04T17:50:36.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.453+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~6QF5EBXAAOAM.1 size=451550 2024-03-04T17:50:36.458 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.461+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=6QF5EBXAAOAM tag=NFR0FNPAMT5G type=write size=2548702 2024-03-04T17:50:36.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.465+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~HH2IGR3FSP1I.0 size=1048576 2024-03-04T17:50:36.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.469+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~HH2IGR3FSP1I.1 size=1048576 2024-03-04T17:50:36.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.473+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~HH2IGR3FSP1I.2 size=65529 2024-03-04T17:50:36.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.473+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=HH2IGR3FSP1I tag=OXFDWPMNVFGD type=write size=3211257 2024-03-04T17:50:36.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.477+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=8T18IBIK5FWH type=del 2024-03-04T17:50:36.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.481+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=5JS5M38PAZKJ type=del size=1180835 2024-03-04T17:50:36.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.485+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=VJCU3OS8VMVC type=del 2024-03-04T17:50:36.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.489+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=XQJ6O4S9WEUB type=del size=1901706 2024-03-04T17:50:36.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.493+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~TJZSHAOZQ4MU.0 size=944760 2024-03-04T17:50:36.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.497+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=TJZSHAOZQ4MU tag=SPG169RPOGMD type=write size=1993336 2024-03-04T17:50:36.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.505+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~QSOUFTA0HTVR.0 size=1048576 2024-03-04T17:50:36.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.509+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~QSOUFTA0HTVR.1 size=1022832 2024-03-04T17:50:36.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.509+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=QSOUFTA0HTVR tag=U4UCKV3GRJAE type=write size=3119984 2024-03-04T17:50:36.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.513+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=9MUFJ0QDZQ40 type=del 2024-03-04T17:50:36.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.521+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~GNTXOFL2FF6W.0 size=1048576 2024-03-04T17:50:36.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.525+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~GNTXOFL2FF6W.1 size=230927 2024-03-04T17:50:36.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.525+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=GNTXOFL2FF6W tag=AURPXBOSCOGY type=write size=2328079 2024-03-04T17:50:36.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.529+0000 7fda08af98c0 1 < completed write operation key=Z2JQBN84QQIB tag=KZK5WBWQLBQX type=write size=665383 2024-03-04T17:50:36.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.533+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=P1JV2RPS1AS5 type=del size=506795 2024-03-04T17:50:36.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.537+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~NRSBOSD3JRYY.0 size=1048576 2024-03-04T17:50:36.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.541+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~NRSBOSD3JRYY.1 size=1048576 2024-03-04T17:50:36.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.541+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~NRSBOSD3JRYY.2 size=883934 2024-03-04T17:50:36.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.549+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=NRSBOSD3JRYY tag=OJTTKUBYQR7U type=write size=4029662 2024-03-04T17:50:36.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.557+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~0KTBRF28D9P5.0 size=1048576 2024-03-04T17:50:36.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.557+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~0KTBRF28D9P5.1 size=427026 2024-03-04T17:50:36.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.557+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=0KTBRF28D9P5 tag=GHMSGUWDDHGB type=write size=2524178 2024-03-04T17:50:36.561 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.561+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=2YFLS70KBP9W type=del 2024-03-04T17:50:36.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.569+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~D45NKINJL64H.0 size=1048576 2024-03-04T17:50:36.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.573+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~D45NKINJL64H.1 size=1048576 2024-03-04T17:50:36.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.577+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~D45NKINJL64H.2 size=433816 2024-03-04T17:50:36.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.577+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=D45NKINJL64H tag=8DAHZX97JIQI type=write size=3579544 2024-03-04T17:50:36.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.581+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=AXKL7L0SISYL type=del 2024-03-04T17:50:36.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.585+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=N5HI6PDXUVQ2 type=del size=1358115 2024-03-04T17:50:36.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.589+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~QHA7GOXM4P59.0 size=39886 2024-03-04T17:50:36.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.593+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=QHA7GOXM4P59 tag=5679IM655MYA type=write size=1088462 2024-03-04T17:50:36.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.597+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=H35ZLS6M1X72 type=del size=3984594 2024-03-04T17:50:36.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.601+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=393PL2K1VDZI type=del 2024-03-04T17:50:36.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.609+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~M80V49SP5R9O.0 size=1048576 2024-03-04T17:50:36.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.613+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~M80V49SP5R9O.1 size=516873 2024-03-04T17:50:36.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.613+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=M80V49SP5R9O tag=URGZK3G9RQ3P type=write size=2614025 2024-03-04T17:50:36.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.621+0000 7fda08af98c0 1 < suggested remove operation key=JA4JK99I0942 tag=QOQMA48AC7Q5 type=del 2024-03-04T17:50:36.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.625+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=0CKLSYX2WTBR type=del size=1390241 2024-03-04T17:50:36.626 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.629+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~YEDGRGIFUM5N.0 size=1048576 2024-03-04T17:50:36.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.629+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~YEDGRGIFUM5N.1 size=1048576 2024-03-04T17:50:36.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.633+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~YEDGRGIFUM5N.2 size=58783 2024-03-04T17:50:36.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.641+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=YEDGRGIFUM5N tag=RE89QWI01APC type=write size=3204511 2024-03-04T17:50:36.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.645+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=K36C1JW4G5CP type=del size=1364183 2024-03-04T17:50:36.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.649+0000 7fda08af98c0 1 < completed write operation key=BYQFZSJ3WH07 tag=MFYZA5ZMV8EK type=write size=937248 2024-03-04T17:50:36.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.653+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=AG5QBMNH3LK6 type=del size=3475906 2024-03-04T17:50:36.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.665+0000 7fda08af98c0 1 < suggested update operation key=2PA8JDZUOLO5 tag=B0QSJFSHOKIA type=del size=2565758 2024-03-04T17:50:36.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.669+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=A36KZ6KZT2ZH type=del size=2218856 2024-03-04T17:50:36.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.673+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=110HWH66PYK5 type=del 2024-03-04T17:50:36.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.677+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=6EWY65FB36ON type=del size=3062167 2024-03-04T17:50:36.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.681+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~HH2IGR3FSP1I.0 size=1048576 2024-03-04T17:50:36.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.685+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~HH2IGR3FSP1I.1 size=1048576 2024-03-04T17:50:36.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.685+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~HH2IGR3FSP1I.2 size=65529 2024-03-04T17:50:36.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.693+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=HH2IGR3FSP1I tag=OXFDWPMNVFGD type=write size=3211257 2024-03-04T17:50:36.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.697+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=U2YT39UKUC6L type=del size=358015 2024-03-04T17:50:36.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.701+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=5JS5M38PAZKJ type=del 2024-03-04T17:50:36.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.705+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=9T0F7N4AYJBM type=write size=550123 2024-03-04T17:50:36.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.709+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=XQJ6O4S9WEUB type=del size=1901706 2024-03-04T17:50:36.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.717+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~COMDV5S1UOUY.0 size=1048576 2024-03-04T17:50:36.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.717+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~COMDV5S1UOUY.1 size=272809 2024-03-04T17:50:36.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.717+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=COMDV5S1UOUY tag=Z3TREHVKERWS type=write size=2369961 2024-03-04T17:50:36.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.721+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~QSOUFTA0HTVR.0 size=1048576 2024-03-04T17:50:36.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.725+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~QSOUFTA0HTVR.1 size=1022832 2024-03-04T17:50:36.729 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.733+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=QSOUFTA0HTVR tag=U4UCKV3GRJAE type=write size=3119984 2024-03-04T17:50:36.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.737+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~I6KWOJLRKTEM.0 size=467332 2024-03-04T17:50:36.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.737+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=I6KWOJLRKTEM tag=XQLI4STE64X9 type=write size=1515908 2024-03-04T17:50:36.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.741+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~GNTXOFL2FF6W.0 size=1048576 2024-03-04T17:50:36.744 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.745+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~GNTXOFL2FF6W.1 size=230927 2024-03-04T17:50:36.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.753+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=GNTXOFL2FF6W tag=AURPXBOSCOGY type=write size=2328079 2024-03-04T17:50:36.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.757+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~IQ857P03R0M3.0 size=1048576 2024-03-04T17:50:36.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.761+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~IQ857P03R0M3.1 size=564157 2024-03-04T17:50:36.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.761+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=IQ857P03R0M3 tag=NGCKH1162SWV type=write size=2661309 2024-03-04T17:50:36.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.765+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=P1JV2RPS1AS5 type=del 2024-03-04T17:50:36.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.773+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~U7BN7JLH4ELH.0 size=1048576 2024-03-04T17:50:36.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.777+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~U7BN7JLH4ELH.1 size=1048576 2024-03-04T17:50:36.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.777+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~U7BN7JLH4ELH.2 size=485096 2024-03-04T17:50:36.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.777+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=U7BN7JLH4ELH tag=GBLVZ8KAXMYJ type=write size=3630824 2024-03-04T17:50:36.780 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.781+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~0KTBRF28D9P5.0 size=1048576 2024-03-04T17:50:36.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.785+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~0KTBRF28D9P5.1 size=427026 2024-03-04T17:50:36.789 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.789+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=0KTBRF28D9P5 tag=GHMSGUWDDHGB type=write size=2524178 2024-03-04T17:50:36.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.793+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=GL9PNF8IS4NY type=write size=880828 2024-03-04T17:50:36.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.797+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~D45NKINJL64H.0 size=1048576 2024-03-04T17:50:36.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.801+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~D45NKINJL64H.1 size=1048576 2024-03-04T17:50:36.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.805+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~D45NKINJL64H.2 size=433816 2024-03-04T17:50:36.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.809+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=D45NKINJL64H tag=8DAHZX97JIQI type=write size=3579544 2024-03-04T17:50:36.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.817+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~VP26YPULRPAU.0 size=1048576 2024-03-04T17:50:36.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.821+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~VP26YPULRPAU.1 size=1048576 2024-03-04T17:50:36.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.821+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~VP26YPULRPAU.2 size=328617 2024-03-04T17:50:36.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.821+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=VP26YPULRPAU tag=WD0Q000P55H4 type=write size=3474345 2024-03-04T17:50:36.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.825+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=N5HI6PDXUVQ2 type=del 2024-03-04T17:50:36.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.833+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~O33YBH1ZNPVQ.0 size=1048576 2024-03-04T17:50:36.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.837+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~O33YBH1ZNPVQ.1 size=1048576 2024-03-04T17:50:36.838 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.841+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~O33YBH1ZNPVQ.2 size=966055 2024-03-04T17:50:36.838 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.841+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=O33YBH1ZNPVQ tag=XJZOIE0L06EI type=write size=4111783 2024-03-04T17:50:36.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.849+0000 7fda08af98c0 1 < suggested update operation key=FMPZ2548LBBF tag=H35ZLS6M1X72 type=del size=3984594 2024-03-04T17:50:36.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.849+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=OZA4IFAAIPXR type=del size=3799725 2024-03-04T17:50:36.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.853+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~M80V49SP5R9O.0 size=1048576 2024-03-04T17:50:36.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.857+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~M80V49SP5R9O.1 size=516873 2024-03-04T17:50:36.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.865+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=M80V49SP5R9O tag=URGZK3G9RQ3P type=write size=2614025 2024-03-04T17:50:36.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.869+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=IRF5PETQIMZ4 type=del size=910206 2024-03-04T17:50:36.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.877+0000 7fda08af98c0 1 < suggested update operation key=Z3J7W9V8HQKH tag=0CKLSYX2WTBR type=del size=1390241 2024-03-04T17:50:36.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.881+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~414HVVSHFCLM.0 size=1048576 2024-03-04T17:50:36.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.885+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~414HVVSHFCLM.1 size=1048576 2024-03-04T17:50:36.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.889+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~414HVVSHFCLM.2 size=730255 2024-03-04T17:50:36.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.889+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=414HVVSHFCLM tag=P36DABAPQRA3 type=write size=3875983 2024-03-04T17:50:36.896 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.897+0000 7fda08af98c0 1 < suggested update operation key=V1LDB2HXZ7EF tag=K36C1JW4G5CP type=del size=1364183 2024-03-04T17:50:36.900 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.901+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=277Q4K6JAJKY type=del size=846479 2024-03-04T17:50:36.904 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.905+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=AG5QBMNH3LK6 type=del 2024-03-04T17:50:36.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.909+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=A9CSTY7TGBLV type=del size=3104936 2024-03-04T17:50:36.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.913+0000 7fda08af98c0 -1 < failed to complete operation key=Z3J7W9V8HQKH tag=A36KZ6KZT2ZH type=del size=2218856: (22) Invalid argument 2024-03-04T17:50:36.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.917+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=HSQX8M60NFFV type=del size=1276398 2024-03-04T17:50:36.921 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.921+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=6EWY65FB36ON type=del 2024-03-04T17:50:36.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.933+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~QIYCRA63VOET.0 size=1048576 2024-03-04T17:50:36.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.933+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~QIYCRA63VOET.1 size=164677 2024-03-04T17:50:36.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.933+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=QIYCRA63VOET tag=BRJHEAI05P1T type=write size=2261829 2024-03-04T17:50:36.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.937+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=U2YT39UKUC6L type=del 2024-03-04T17:50:36.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.941+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=EA7ZE0JN9084 type=del size=1982565 2024-03-04T17:50:36.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.945+0000 7fda08af98c0 1 < completed write operation key=AO3QS6CM7I5D tag=9T0F7N4AYJBM type=write size=550123 2024-03-04T17:50:36.953 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.953+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~10YMI44S3N0I.0 size=1048576 2024-03-04T17:50:36.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.957+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~10YMI44S3N0I.1 size=531883 2024-03-04T17:50:36.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.957+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=10YMI44S3N0I tag=JZE2OM535MMH type=write size=2629035 2024-03-04T17:50:36.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.961+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~COMDV5S1UOUY.0 size=1048576 2024-03-04T17:50:36.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.965+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~COMDV5S1UOUY.1 size=272809 2024-03-04T17:50:36.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.973+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=COMDV5S1UOUY tag=Z3TREHVKERWS type=write size=2369961 2024-03-04T17:50:36.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.973+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=JFIUP4WGW9IH type=write size=478260 2024-03-04T17:50:36.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.977+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~I6KWOJLRKTEM.0 size=467332 2024-03-04T17:50:36.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.985+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=I6KWOJLRKTEM tag=XQLI4STE64X9 type=write size=1515908 2024-03-04T17:50:36.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.993+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~QKKLVMOD2EKG.0 size=1048576 2024-03-04T17:50:36.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:36.997+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~QKKLVMOD2EKG.1 size=1048576 2024-03-04T17:50:36.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.001+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~QKKLVMOD2EKG.2 size=71091 2024-03-04T17:50:36.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.001+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=QKKLVMOD2EKG tag=5QTQYXLVETDJ type=write size=3216819 2024-03-04T17:50:37.002 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.005+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~IQ857P03R0M3.0 size=1048576 2024-03-04T17:50:37.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.005+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~IQ857P03R0M3.1 size=564157 2024-03-04T17:50:37.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.017+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=IQ857P03R0M3 tag=NGCKH1162SWV type=write size=2661309 2024-03-04T17:50:37.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.025+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~PY37EA3CQ4WO.0 size=1048576 2024-03-04T17:50:37.025 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.029+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~PY37EA3CQ4WO.1 size=1048576 2024-03-04T17:50:37.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.029+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~PY37EA3CQ4WO.2 size=709820 2024-03-04T17:50:37.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.029+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=PY37EA3CQ4WO tag=HOVR3WCA6ZIW type=write size=3855548 2024-03-04T17:50:37.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.033+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~U7BN7JLH4ELH.0 size=1048576 2024-03-04T17:50:37.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.037+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~U7BN7JLH4ELH.1 size=1048576 2024-03-04T17:50:37.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.041+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~U7BN7JLH4ELH.2 size=485096 2024-03-04T17:50:37.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.045+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=U7BN7JLH4ELH tag=GBLVZ8KAXMYJ type=write size=3630824 2024-03-04T17:50:37.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.049+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=LJ8HUYKDL0B7 type=del size=1174010 2024-03-04T17:50:37.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.057+0000 7fda08af98c0 1 < suggested update operation key=Z2JQBN84QQIB tag=GL9PNF8IS4NY type=write size=880828 2024-03-04T17:50:37.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.065+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~K3YT6OPJHZMQ.0 size=930983 2024-03-04T17:50:37.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.065+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=K3YT6OPJHZMQ tag=KDGY9X2FI8QG type=write size=1979559 2024-03-04T17:50:37.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.069+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~VP26YPULRPAU.0 size=1048576 2024-03-04T17:50:37.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.069+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~VP26YPULRPAU.1 size=1048576 2024-03-04T17:50:37.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.073+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~VP26YPULRPAU.2 size=328617 2024-03-04T17:50:37.077 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.077+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=VP26YPULRPAU tag=WD0Q000P55H4 type=write size=3474345 2024-03-04T17:50:37.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.081+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=LC1D3M3QAB44 type=del size=1917153 2024-03-04T17:50:37.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.085+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~O33YBH1ZNPVQ.0 size=1048576 2024-03-04T17:50:37.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.089+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~O33YBH1ZNPVQ.1 size=1048576 2024-03-04T17:50:37.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.093+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~O33YBH1ZNPVQ.2 size=966055 2024-03-04T17:50:37.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.097+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=O33YBH1ZNPVQ tag=XJZOIE0L06EI type=write size=4111783 2024-03-04T17:50:37.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.101+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=OY67M706BOTA type=del size=2509527 2024-03-04T17:50:37.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.109+0000 7fda08af98c0 1 < suggested remove operation key=2PA8JDZUOLO5 tag=OZA4IFAAIPXR type=del 2024-03-04T17:50:37.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.117+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~SVZ6GJS9P1M7.0 size=494899 2024-03-04T17:50:37.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.117+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=SVZ6GJS9P1M7 tag=GUJ2FZXZIACI type=write size=1543475 2024-03-04T17:50:37.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.121+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=IRF5PETQIMZ4 type=del 2024-03-04T17:50:37.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.125+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=SVA0B463VX4F type=del size=997032 2024-03-04T17:50:37.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.129+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~414HVVSHFCLM.0 size=1048576 2024-03-04T17:50:37.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.129+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~414HVVSHFCLM.1 size=1048576 2024-03-04T17:50:37.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.133+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~414HVVSHFCLM.2 size=730255 2024-03-04T17:50:37.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.141+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=414HVVSHFCLM tag=P36DABAPQRA3 type=write size=3875983 2024-03-04T17:50:37.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.145+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=0R1K41NQFJT1 type=del size=5210 2024-03-04T17:50:37.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.149+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=277Q4K6JAJKY type=del 2024-03-04T17:50:37.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.153+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=KD54QCSBQ9OX type=del size=2077382 2024-03-04T17:50:37.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.157+0000 7fda08af98c0 1 < canceled operation key=RXVSQVVNXVF0 tag=A9CSTY7TGBLV type=del size=3104936 2024-03-04T17:50:37.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.157+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=30A72VC83BHP type=del size=1943789 2024-03-04T17:50:37.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.161+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=HSQX8M60NFFV type=del 2024-03-04T17:50:37.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.165+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=M77V7E0UW6BA type=write size=381834 2024-03-04T17:50:37.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.169+0000 7fda08af98c0 1 < canceled part key=_multipart_750NSVFBS03A.2~QIYCRA63VOET.0 size=1048576 2024-03-04T17:50:37.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.169+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~QIYCRA63VOET.1 size=164677 2024-03-04T17:50:37.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.177+0000 7fda08af98c0 1 < canceled multipart upload key=750NSVFBS03A upload=QIYCRA63VOET tag=BRJHEAI05P1T type=write size=2261829 2024-03-04T17:50:37.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.181+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=1RFHQEUVLEWT type=del size=4111455 2024-03-04T17:50:37.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.185+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=EA7ZE0JN9084 type=del 2024-03-04T17:50:37.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.189+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=A4B5L0U4DUQ8 type=del size=1701527 2024-03-04T17:50:37.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.193+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~10YMI44S3N0I.0 size=1048576 2024-03-04T17:50:37.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.197+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~10YMI44S3N0I.1 size=531883 2024-03-04T17:50:37.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.201+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=10YMI44S3N0I tag=JZE2OM535MMH type=write size=2629035 2024-03-04T17:50:37.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.209+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~9RUW8I8R1OG8.0 size=308815 2024-03-04T17:50:37.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.209+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=9RUW8I8R1OG8 tag=ZVOJ1DRG15W9 type=write size=1357391 2024-03-04T17:50:37.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.213+0000 7fda08af98c0 1 < completed write operation key=C4B45GL40IMZ tag=JFIUP4WGW9IH type=write size=478260 2024-03-04T17:50:37.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.217+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=PYOBX0WJ5X3S type=del size=2153562 2024-03-04T17:50:37.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.221+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~QKKLVMOD2EKG.0 size=1048576 2024-03-04T17:50:37.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.225+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~QKKLVMOD2EKG.1 size=1048576 2024-03-04T17:50:37.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.225+0000 7fda08af98c0 1 < canceled part key=_multipart_8YL7C9PI58TK.2~QKKLVMOD2EKG.2 size=71091 2024-03-04T17:50:37.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.233+0000 7fda08af98c0 1 < canceled multipart upload key=8YL7C9PI58TK upload=QKKLVMOD2EKG tag=5QTQYXLVETDJ type=write size=3216819 2024-03-04T17:50:37.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.237+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=63TMD3G4QGK1 type=write size=45475 2024-03-04T17:50:37.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.241+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~PY37EA3CQ4WO.0 size=1048576 2024-03-04T17:50:37.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.241+0000 7fda08af98c0 1 < canceled part key=_multipart_IJSYAVR5SWPS.2~PY37EA3CQ4WO.1 size=1048576 2024-03-04T17:50:37.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.245+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~PY37EA3CQ4WO.2 size=709820 2024-03-04T17:50:37.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.253+0000 7fda08af98c0 1 < canceled multipart upload key=IJSYAVR5SWPS upload=PY37EA3CQ4WO tag=HOVR3WCA6ZIW type=write size=3855548 2024-03-04T17:50:37.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.261+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~33QZXSPOFNPH.0 size=1048576 2024-03-04T17:50:37.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.261+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~33QZXSPOFNPH.1 size=977768 2024-03-04T17:50:37.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.261+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=33QZXSPOFNPH tag=R02FKCJEWDQF type=write size=3074920 2024-03-04T17:50:37.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.269+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=LJ8HUYKDL0B7 type=del 2024-03-04T17:50:37.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.273+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=FP3ORF2XEN1R type=del size=798725 2024-03-04T17:50:37.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.277+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~K3YT6OPJHZMQ.0 size=930983 2024-03-04T17:50:37.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.281+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=K3YT6OPJHZMQ tag=KDGY9X2FI8QG type=write size=1979559 2024-03-04T17:50:37.288 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.289+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~Q7YDBRDJ3XUU.0 size=913142 2024-03-04T17:50:37.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.289+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=Q7YDBRDJ3XUU tag=42YBZE35EAVE type=write size=1961718 2024-03-04T17:50:37.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.297+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=LC1D3M3QAB44 type=del 2024-03-04T17:50:37.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.305+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~BZ083245IHA5.0 size=1048576 2024-03-04T17:50:37.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.305+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~BZ083245IHA5.1 size=1048576 2024-03-04T17:50:37.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.309+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~BZ083245IHA5.2 size=348656 2024-03-04T17:50:37.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.309+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=BZ083245IHA5 tag=VEYB71N6HKWC type=write size=3494384 2024-03-04T17:50:37.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.313+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=OY67M706BOTA type=del 2024-03-04T17:50:37.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.317+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~3RK7VMWLN47R.0 size=1048576 2024-03-04T17:50:37.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.321+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~3RK7VMWLN47R.1 size=1048576 2024-03-04T17:50:37.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.325+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~3RK7VMWLN47R.2 size=828812 2024-03-04T17:50:37.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.325+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=3RK7VMWLN47R tag=81AE3CW8RJ6S type=write size=3974540 2024-03-04T17:50:37.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.329+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~SVZ6GJS9P1M7.0 size=494899 2024-03-04T17:50:37.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.337+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=SVZ6GJS9P1M7 tag=GUJ2FZXZIACI type=write size=1543475 2024-03-04T17:50:37.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.341+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~D02QELXZFNNC.0 size=109384 2024-03-04T17:50:37.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.341+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=D02QELXZFNNC tag=BWFRN6DW7TW0 type=write size=1157960 2024-03-04T17:50:37.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.345+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=SVA0B463VX4F type=del 2024-03-04T17:50:37.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.349+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=O2LTU1ML9UDW type=write size=691607 2024-03-04T17:50:37.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.353+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=0R1K41NQFJT1 type=del 2024-03-04T17:50:37.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.361+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~1DE01H7SK2PR.0 size=1048576 2024-03-04T17:50:37.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.365+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~1DE01H7SK2PR.1 size=930521 2024-03-04T17:50:37.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.365+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=1DE01H7SK2PR tag=32E24L9OMKDK type=write size=3027673 2024-03-04T17:50:37.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.369+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=KD54QCSBQ9OX type=del 2024-03-04T17:50:37.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.377+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~6WHDQ6QSTVSG.0 size=1048576 2024-03-04T17:50:37.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.381+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~6WHDQ6QSTVSG.1 size=814703 2024-03-04T17:50:37.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.381+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=6WHDQ6QSTVSG tag=A8J4KPMVIQPI type=write size=2911855 2024-03-04T17:50:37.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.385+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=30A72VC83BHP type=del 2024-03-04T17:50:37.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.393+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~R40A8BO75Z91.0 size=167358 2024-03-04T17:50:37.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.393+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=R40A8BO75Z91 tag=IBODCQQA2C22 type=write size=1215934 2024-03-04T17:50:37.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.397+0000 7fda08af98c0 1 < completed write operation key=C1GK00FM2ZYA tag=M77V7E0UW6BA type=write size=381834 2024-03-04T17:50:37.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.401+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~0EWNY85HENV7.0 size=999167 2024-03-04T17:50:37.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.401+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=0EWNY85HENV7 tag=LG1CYUNT20HV type=write size=2047743 2024-03-04T17:50:37.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.405+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=1RFHQEUVLEWT type=del 2024-03-04T17:50:37.412 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.413+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~GS0H32N34IUK.0 size=1048576 2024-03-04T17:50:37.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.417+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~GS0H32N34IUK.1 size=694131 2024-03-04T17:50:37.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.417+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=GS0H32N34IUK tag=4A4YMKPZR003 type=write size=2791283 2024-03-04T17:50:37.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.425+0000 7fda08af98c0 1 < suggested update operation key=IJSYAVR5SWPS tag=A4B5L0U4DUQ8 type=del size=1701527 2024-03-04T17:50:37.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.429+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=YC5LZXYRD6JO type=del size=1987692 2024-03-04T17:50:37.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.433+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~9RUW8I8R1OG8.0 size=308815 2024-03-04T17:50:37.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.441+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=9RUW8I8R1OG8 tag=ZVOJ1DRG15W9 type=write size=1357391 2024-03-04T17:50:37.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.445+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=MUB933Y3H8B5 type=del size=2092166 2024-03-04T17:50:37.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.453+0000 7fda08af98c0 1 < suggested update operation key=Z3J7W9V8HQKH tag=PYOBX0WJ5X3S type=del size=2153562 2024-03-04T17:50:37.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.461+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~GDIXUWTHXCIP.0 size=1048576 2024-03-04T17:50:37.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.465+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~GDIXUWTHXCIP.1 size=955341 2024-03-04T17:50:37.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.465+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=GDIXUWTHXCIP tag=SVU36E6ZRIXQ type=write size=3052493 2024-03-04T17:50:37.469 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.469+0000 7fda08af98c0 1 < completed write operation key=8YL7C9PI58TK tag=63TMD3G4QGK1 type=write size=45475 2024-03-04T17:50:37.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.477+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~D10MGP1TXK2C.0 size=1048576 2024-03-04T17:50:37.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.481+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~D10MGP1TXK2C.1 size=435501 2024-03-04T17:50:37.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.481+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=D10MGP1TXK2C tag=02STQBGDZ9UJ type=write size=2532653 2024-03-04T17:50:37.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.485+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~33QZXSPOFNPH.0 size=1048576 2024-03-04T17:50:37.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.489+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~33QZXSPOFNPH.1 size=977768 2024-03-04T17:50:37.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.493+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=33QZXSPOFNPH tag=R02FKCJEWDQF type=write size=3074920 2024-03-04T17:50:37.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.501+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~UPDUNJYODHA1.0 size=1048576 2024-03-04T17:50:37.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.501+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~UPDUNJYODHA1.1 size=1048576 2024-03-04T17:50:37.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.505+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~UPDUNJYODHA1.2 size=857451 2024-03-04T17:50:37.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.505+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=UPDUNJYODHA1 tag=THS9XE31EBHD type=write size=4003179 2024-03-04T17:50:37.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.509+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=FP3ORF2XEN1R type=del 2024-03-04T17:50:37.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.513+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=VM7ACGEUH33V type=del size=920676 2024-03-04T17:50:37.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.513+0000 7fda08af98c0 1 < canceled part key=_multipart_8YL7C9PI58TK.2~Q7YDBRDJ3XUU.0 size=913142 2024-03-04T17:50:37.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.521+0000 7fda08af98c0 1 < canceled multipart upload key=8YL7C9PI58TK upload=Q7YDBRDJ3XUU tag=42YBZE35EAVE type=write size=1961718 2024-03-04T17:50:37.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.525+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~RNVGK64GXP7J.0 size=1048576 2024-03-04T17:50:37.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.529+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~RNVGK64GXP7J.1 size=84996 2024-03-04T17:50:37.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.529+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=RNVGK64GXP7J tag=0XKQ1RLRJ9GW type=write size=2182148 2024-03-04T17:50:37.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.529+0000 7fda08af98c0 1 < canceled part key=_multipart_GDGTJZ21TG32.2~BZ083245IHA5.0 size=1048576 2024-03-04T17:50:37.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.533+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~BZ083245IHA5.1 size=1048576 2024-03-04T17:50:37.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.533+0000 7fda08af98c0 1 < canceled part key=_multipart_GDGTJZ21TG32.2~BZ083245IHA5.2 size=348656 2024-03-04T17:50:37.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.537+0000 7fda08af98c0 1 < canceled multipart upload key=GDGTJZ21TG32 upload=BZ083245IHA5 tag=VEYB71N6HKWC type=write size=3494384 2024-03-04T17:50:37.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.541+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=1I68FUACNJV2 type=del size=114866 2024-03-04T17:50:37.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.545+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~3RK7VMWLN47R.0 size=1048576 2024-03-04T17:50:37.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.549+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~3RK7VMWLN47R.1 size=1048576 2024-03-04T17:50:37.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.553+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~3RK7VMWLN47R.2 size=828812 2024-03-04T17:50:37.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.557+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=3RK7VMWLN47R tag=81AE3CW8RJ6S type=write size=3974540 2024-03-04T17:50:37.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.565+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~FJ2EVCY5RBET.0 size=433967 2024-03-04T17:50:37.565 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.565+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=FJ2EVCY5RBET tag=G131BSJ1U7S0 type=write size=1482543 2024-03-04T17:50:37.565 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.565+0000 7fda08af98c0 1 < canceled part key=_multipart_1OZKPHMPV3OQ.2~D02QELXZFNNC.0 size=109384 2024-03-04T17:50:37.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.573+0000 7fda08af98c0 1 < canceled multipart upload key=1OZKPHMPV3OQ upload=D02QELXZFNNC tag=BWFRN6DW7TW0 type=write size=1157960 2024-03-04T17:50:37.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.577+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=2JCV5X3C0K4S type=del size=3857432 2024-03-04T17:50:37.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.581+0000 7fda08af98c0 1 < completed write operation key=8YL7C9PI58TK tag=O2LTU1ML9UDW type=write size=691607 2024-03-04T17:50:37.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.585+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=2T4RCMC29EOX type=del size=717840 2024-03-04T17:50:37.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.589+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~1DE01H7SK2PR.0 size=1048576 2024-03-04T17:50:37.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.593+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~1DE01H7SK2PR.1 size=930521 2024-03-04T17:50:37.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.597+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=1DE01H7SK2PR tag=32E24L9OMKDK type=write size=3027673 2024-03-04T17:50:37.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.601+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=RX513COU0ITF type=write size=19907 2024-03-04T17:50:37.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.605+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~6WHDQ6QSTVSG.0 size=1048576 2024-03-04T17:50:37.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.609+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~6WHDQ6QSTVSG.1 size=814703 2024-03-04T17:50:37.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.613+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=6WHDQ6QSTVSG tag=A8J4KPMVIQPI type=write size=2911855 2024-03-04T17:50:37.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.621+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~7LQW8JHU38JY.0 size=44652 2024-03-04T17:50:37.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.621+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=7LQW8JHU38JY tag=1X89BYOTY3BZ type=write size=1093228 2024-03-04T17:50:37.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.625+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~R40A8BO75Z91.0 size=167358 2024-03-04T17:50:37.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.629+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=R40A8BO75Z91 tag=IBODCQQA2C22 type=write size=1215934 2024-03-04T17:50:37.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.637+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~2YTXF2RE2O5M.0 size=1048576 2024-03-04T17:50:37.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.641+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~2YTXF2RE2O5M.1 size=505659 2024-03-04T17:50:37.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.641+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=2YTXF2RE2O5M tag=S6RCRUB7N20K type=write size=2602811 2024-03-04T17:50:37.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.645+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~0EWNY85HENV7.0 size=999167 2024-03-04T17:50:37.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.653+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=0EWNY85HENV7 tag=LG1CYUNT20HV type=write size=2047743 2024-03-04T17:50:37.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.657+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=6PKX17CFMMBU type=del size=246894 2024-03-04T17:50:37.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.661+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~GS0H32N34IUK.0 size=1048576 2024-03-04T17:50:37.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.661+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~GS0H32N34IUK.1 size=694131 2024-03-04T17:50:37.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.669+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=GS0H32N34IUK tag=4A4YMKPZR003 type=write size=2791283 2024-03-04T17:50:37.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.677+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~A4XF60G52DHC.0 size=1048576 2024-03-04T17:50:37.677 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.677+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~A4XF60G52DHC.1 size=1048576 2024-03-04T17:50:37.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.681+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~A4XF60G52DHC.2 size=756614 2024-03-04T17:50:37.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.681+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=A4XF60G52DHC tag=ACK36A3I65HF type=write size=3902342 2024-03-04T17:50:37.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.685+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=YC5LZXYRD6JO type=del 2024-03-04T17:50:37.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.689+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=V7E032B1HC9D type=del size=2230349 2024-03-04T17:50:37.689 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.693+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=MUB933Y3H8B5 type=del 2024-03-04T17:50:37.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.697+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~T26LNA1AR6K4.0 size=1048576 2024-03-04T17:50:37.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.701+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~T26LNA1AR6K4.1 size=1048576 2024-03-04T17:50:37.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.701+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~T26LNA1AR6K4.2 size=831120 2024-03-04T17:50:37.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.701+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=T26LNA1AR6K4 tag=PRHAHB0S2GRI type=write size=3976848 2024-03-04T17:50:37.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.705+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~GDIXUWTHXCIP.0 size=1048576 2024-03-04T17:50:37.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.709+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~GDIXUWTHXCIP.1 size=955341 2024-03-04T17:50:37.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.713+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=GDIXUWTHXCIP tag=SVU36E6ZRIXQ type=write size=3052493 2024-03-04T17:50:37.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.721+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=HMDIO3122MBG type=del size=1849922 2024-03-04T17:50:37.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.725+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~D10MGP1TXK2C.0 size=1048576 2024-03-04T17:50:37.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.725+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~D10MGP1TXK2C.1 size=435501 2024-03-04T17:50:37.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.733+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=D10MGP1TXK2C tag=02STQBGDZ9UJ type=write size=2532653 2024-03-04T17:50:37.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.741+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~6FED9F9SAM2M.0 size=1048576 2024-03-04T17:50:37.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.741+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~6FED9F9SAM2M.1 size=1048576 2024-03-04T17:50:37.743 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.745+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~6FED9F9SAM2M.2 size=654987 2024-03-04T17:50:37.743 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.745+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=6FED9F9SAM2M tag=SU3L3KC1U2C5 type=write size=3800715 2024-03-04T17:50:37.747 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.749+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~UPDUNJYODHA1.0 size=1048576 2024-03-04T17:50:37.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.753+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~UPDUNJYODHA1.1 size=1048576 2024-03-04T17:50:37.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.753+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~UPDUNJYODHA1.2 size=857451 2024-03-04T17:50:37.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.761+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=UPDUNJYODHA1 tag=THS9XE31EBHD type=write size=4003179 2024-03-04T17:50:37.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.765+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=E9V8EU943XM3 type=write size=112601 2024-03-04T17:50:37.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.769+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=VM7ACGEUH33V type=del 2024-03-04T17:50:37.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.777+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~DV0BDWVROVW6.0 size=975462 2024-03-04T17:50:37.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.777+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=DV0BDWVROVW6 tag=G8YA78XG1WUZ type=write size=2024038 2024-03-04T17:50:37.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.781+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~RNVGK64GXP7J.0 size=1048576 2024-03-04T17:50:37.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.781+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~RNVGK64GXP7J.1 size=84996 2024-03-04T17:50:37.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.789+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=RNVGK64GXP7J tag=0XKQ1RLRJ9GW type=write size=2182148 2024-03-04T17:50:37.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.793+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=R4F4MHGGF4MD type=del size=3654986 2024-03-04T17:50:37.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.801+0000 7fda08af98c0 1 < suggested remove operation key=1OZKPHMPV3OQ tag=1I68FUACNJV2 type=del 2024-03-04T17:50:37.805 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.809+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~RXHM5NMRN4JS.0 size=1048576 2024-03-04T17:50:37.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.809+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~RXHM5NMRN4JS.1 size=847141 2024-03-04T17:50:37.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.809+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=RXHM5NMRN4JS tag=5Q64NRJLH2UL type=write size=2944293 2024-03-04T17:50:37.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.813+0000 7fda08af98c0 1 < canceled part key=_multipart_3UWSL7NJLACY.2~FJ2EVCY5RBET.0 size=433967 2024-03-04T17:50:37.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.817+0000 7fda08af98c0 1 < canceled multipart upload key=3UWSL7NJLACY upload=FJ2EVCY5RBET tag=G131BSJ1U7S0 type=write size=1482543 2024-03-04T17:50:37.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.825+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~4BH6Z79DQS0U.0 size=953514 2024-03-04T17:50:37.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.825+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=4BH6Z79DQS0U tag=S4TZVIYYTOQ2 type=write size=2002090 2024-03-04T17:50:37.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.829+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=2JCV5X3C0K4S type=del 2024-03-04T17:50:37.830 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.833+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=ARZ0U6TNU02B type=del size=2274430 2024-03-04T17:50:37.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.837+0000 7fda08af98c0 1 < canceled operation key=FMPZ2548LBBF tag=2T4RCMC29EOX type=del size=717840 2024-03-04T17:50:37.838 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.841+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=WZT2ETXFFCRJ type=del size=1372581 2024-03-04T17:50:37.842 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.845+0000 7fda08af98c0 1 < completed write operation key=G0BHMSCO0XLF tag=RX513COU0ITF type=write size=19907 2024-03-04T17:50:37.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.849+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=LAL7ORR0US6T type=del size=1050355 2024-03-04T17:50:37.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.853+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~7LQW8JHU38JY.0 size=44652 2024-03-04T17:50:37.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.857+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=7LQW8JHU38JY tag=1X89BYOTY3BZ type=write size=1093228 2024-03-04T17:50:37.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.861+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=10DEPNJEG6OM type=del size=2663307 2024-03-04T17:50:37.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.869+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~2YTXF2RE2O5M.0 size=1048576 2024-03-04T17:50:37.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.869+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~2YTXF2RE2O5M.1 size=505659 2024-03-04T17:50:37.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.877+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=2YTXF2RE2O5M tag=S6RCRUB7N20K type=write size=2602811 2024-03-04T17:50:37.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.881+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=J5COUSEZZ3HN type=del size=2488534 2024-03-04T17:50:37.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.885+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=6PKX17CFMMBU type=del 2024-03-04T17:50:37.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.893+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~YG5FRNR6YQE4.0 size=30361 2024-03-04T17:50:37.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.893+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=YG5FRNR6YQE4 tag=PJG2FRBU07K9 type=write size=1078937 2024-03-04T17:50:37.894 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.897+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~A4XF60G52DHC.0 size=1048576 2024-03-04T17:50:37.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.901+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~A4XF60G52DHC.1 size=1048576 2024-03-04T17:50:37.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.901+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~A4XF60G52DHC.2 size=756614 2024-03-04T17:50:37.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.909+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=A4XF60G52DHC tag=ACK36A3I65HF type=write size=3902342 2024-03-04T17:50:37.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.917+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~SZ6PSN6IP0GR.0 size=1048576 2024-03-04T17:50:37.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.917+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~SZ6PSN6IP0GR.1 size=1048576 2024-03-04T17:50:37.920 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.921+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~SZ6PSN6IP0GR.2 size=722032 2024-03-04T17:50:37.920 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.921+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=SZ6PSN6IP0GR tag=VK8F6ND8HY6B type=write size=3867760 2024-03-04T17:50:37.927 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.929+0000 7fda08af98c0 1 < suggested remove operation key=GDGTJZ21TG32 tag=V7E032B1HC9D type=del 2024-03-04T17:50:37.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.933+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=Z4W3L38H1T1U type=write size=44328 2024-03-04T17:50:37.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.937+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~T26LNA1AR6K4.0 size=1048576 2024-03-04T17:50:37.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.941+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~T26LNA1AR6K4.1 size=1048576 2024-03-04T17:50:37.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.941+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~T26LNA1AR6K4.2 size=831120 2024-03-04T17:50:37.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.949+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=T26LNA1AR6K4 tag=PRHAHB0S2GRI type=write size=3976848 2024-03-04T17:50:37.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.953+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=6W2VNPUI34B4 type=del size=3303381 2024-03-04T17:50:37.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.961+0000 7fda08af98c0 1 < suggested update operation key=RXVSQVVNXVF0 tag=HMDIO3122MBG type=del size=1849922 2024-03-04T17:50:37.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.965+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=PIJ08XLIYO51 type=write size=985958 2024-03-04T17:50:37.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.969+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~6FED9F9SAM2M.0 size=1048576 2024-03-04T17:50:37.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.969+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~6FED9F9SAM2M.1 size=1048576 2024-03-04T17:50:37.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.973+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~6FED9F9SAM2M.2 size=654987 2024-03-04T17:50:37.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.981+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=6FED9F9SAM2M tag=SU3L3KC1U2C5 type=write size=3800715 2024-03-04T17:50:37.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.985+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~IES2WGIO53XN.0 size=1048576 2024-03-04T17:50:37.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.989+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~IES2WGIO53XN.1 size=1048576 2024-03-04T17:50:37.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.993+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~IES2WGIO53XN.2 size=553738 2024-03-04T17:50:37.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:37.993+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=IES2WGIO53XN tag=2FMFNG3WSVLF type=write size=3699466 2024-03-04T17:50:37.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.001+0000 7fda08af98c0 1 < suggested update operation key=09KBCA2XURTF tag=E9V8EU943XM3 type=write size=112601 2024-03-04T17:50:38.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.009+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~6N0YMAM3JOIV.0 size=1048576 2024-03-04T17:50:38.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.009+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~6N0YMAM3JOIV.1 size=340185 2024-03-04T17:50:38.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.009+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=6N0YMAM3JOIV tag=QQISPJUZ75F3 type=write size=2437337 2024-03-04T17:50:38.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.017+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~DV0BDWVROVW6.0 size=975462 2024-03-04T17:50:38.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.021+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=DV0BDWVROVW6 tag=G8YA78XG1WUZ type=write size=2024038 2024-03-04T17:50:38.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.025+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=99VQKMI2OAKK type=write size=361934 2024-03-04T17:50:38.028 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.029+0000 7fda08af98c0 -1 < failed to complete operation key=09KBCA2XURTF tag=R4F4MHGGF4MD type=del size=3654986: (22) Invalid argument 2024-03-04T17:50:38.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.033+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=WY9ZE0PT32OS type=write size=875337 2024-03-04T17:50:38.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.037+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~RXHM5NMRN4JS.0 size=1048576 2024-03-04T17:50:38.039 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.041+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~RXHM5NMRN4JS.1 size=847141 2024-03-04T17:50:38.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.049+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=RXHM5NMRN4JS tag=5Q64NRJLH2UL type=write size=2944293 2024-03-04T17:50:38.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.053+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~J59IM3JPEABG.0 size=502046 2024-03-04T17:50:38.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.053+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=J59IM3JPEABG tag=GP9A2B9VJ4HS type=write size=1550622 2024-03-04T17:50:38.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.057+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~4BH6Z79DQS0U.0 size=953514 2024-03-04T17:50:38.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.065+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=4BH6Z79DQS0U tag=S4TZVIYYTOQ2 type=write size=2002090 2024-03-04T17:50:38.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.073+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=EESE7HHUX2I2 type=del size=4084624 2024-03-04T17:50:38.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.077+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=ARZ0U6TNU02B type=del 2024-03-04T17:50:38.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.081+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=GNN2QBV2AXW6 type=del size=1347188 2024-03-04T17:50:38.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.085+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=WZT2ETXFFCRJ type=del 2024-03-04T17:50:38.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.089+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=2YYNHOSHPUT8 type=del size=999662 2024-03-04T17:50:38.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.093+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=LAL7ORR0US6T type=del 2024-03-04T17:50:38.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.101+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~GI5B06Q9ZRRR.0 size=848710 2024-03-04T17:50:38.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.101+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=GI5B06Q9ZRRR tag=237XLZ5B2XJ4 type=write size=1897286 2024-03-04T17:50:38.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.105+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=10DEPNJEG6OM type=del 2024-03-04T17:50:38.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.105+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=7CTMCFWVW813 type=del size=684264 2024-03-04T17:50:38.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.113+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=J5COUSEZZ3HN type=del 2024-03-04T17:50:38.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.117+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=KHL46VPFUGGP type=del size=3334048 2024-03-04T17:50:38.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.121+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~YG5FRNR6YQE4.0 size=30361 2024-03-04T17:50:38.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.125+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=YG5FRNR6YQE4 tag=PJG2FRBU07K9 type=write size=1078937 2024-03-04T17:50:38.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.129+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=VEB44VAK6Q1L type=del size=3067952 2024-03-04T17:50:38.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.133+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~SZ6PSN6IP0GR.0 size=1048576 2024-03-04T17:50:38.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.137+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~SZ6PSN6IP0GR.1 size=1048576 2024-03-04T17:50:38.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.141+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~SZ6PSN6IP0GR.2 size=722032 2024-03-04T17:50:38.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.145+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=SZ6PSN6IP0GR tag=VK8F6ND8HY6B type=write size=3867760 2024-03-04T17:50:38.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.149+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=XCVBADTB82KH type=del size=255483 2024-03-04T17:50:38.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.153+0000 7fda08af98c0 1 < completed write operation key=750NSVFBS03A tag=Z4W3L38H1T1U type=write size=44328 2024-03-04T17:50:38.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.161+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=7Y2P3139WUR5 type=del size=2678877 2024-03-04T17:50:38.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.165+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=6W2VNPUI34B4 type=del 2024-03-04T17:50:38.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.169+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=I30OJZMGZ5XF type=write size=131186 2024-03-04T17:50:38.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.173+0000 7fda08af98c0 1 < completed write operation key=XG2XYBCMUNY5 tag=PIJ08XLIYO51 type=write size=985958 2024-03-04T17:50:38.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.177+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=7WNKDTO67BM3 type=del size=3582489 2024-03-04T17:50:38.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.181+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~IES2WGIO53XN.0 size=1048576 2024-03-04T17:50:38.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.185+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~IES2WGIO53XN.1 size=1048576 2024-03-04T17:50:38.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.185+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~IES2WGIO53XN.2 size=553738 2024-03-04T17:50:38.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.193+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=IES2WGIO53XN tag=2FMFNG3WSVLF type=write size=3699466 2024-03-04T17:50:38.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.201+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~YI9MW1S6JFDQ.0 size=224582 2024-03-04T17:50:38.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.201+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=YI9MW1S6JFDQ tag=IWOLD3JEIHZ5 type=write size=1273158 2024-03-04T17:50:38.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.205+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~6N0YMAM3JOIV.0 size=1048576 2024-03-04T17:50:38.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.205+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~6N0YMAM3JOIV.1 size=340185 2024-03-04T17:50:38.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.213+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=6N0YMAM3JOIV tag=QQISPJUZ75F3 type=write size=2437337 2024-03-04T17:50:38.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.221+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~KLP5ZRHM41CE.0 size=801796 2024-03-04T17:50:38.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.221+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=KLP5ZRHM41CE tag=4N2PD708W4MF type=write size=1850372 2024-03-04T17:50:38.225 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.229+0000 7fda08af98c0 1 < suggested update operation key=C4B45GL40IMZ tag=99VQKMI2OAKK type=write size=361934 2024-03-04T17:50:38.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.229+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=M8C3H2G2MWDA type=write size=41214 2024-03-04T17:50:38.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.233+0000 7fda08af98c0 1 < completed write operation key=ITIVF3UAT9NK tag=WY9ZE0PT32OS type=write size=875337 2024-03-04T17:50:38.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.237+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=8LJTU3SVF2G2 type=del size=809999 2024-03-04T17:50:38.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.241+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~J59IM3JPEABG.0 size=502046 2024-03-04T17:50:38.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.249+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=J59IM3JPEABG tag=GP9A2B9VJ4HS type=write size=1550622 2024-03-04T17:50:38.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.253+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=K97EI2S4J3X3 type=del size=3053606 2024-03-04T17:50:38.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.261+0000 7fda08af98c0 1 < suggested remove operation key=BYQFZSJ3WH07 tag=EESE7HHUX2I2 type=del 2024-03-04T17:50:38.262 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.265+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=JSZ2111BXA81 type=del size=3220564 2024-03-04T17:50:38.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.269+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=GNN2QBV2AXW6 type=del 2024-03-04T17:50:38.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.273+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=RT0PNVVJ9V0J type=del size=2836226 2024-03-04T17:50:38.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.277+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=2YYNHOSHPUT8 type=del 2024-03-04T17:50:38.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.281+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=O54JT7JZGZJR type=del size=2663358 2024-03-04T17:50:38.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.285+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~GI5B06Q9ZRRR.0 size=848710 2024-03-04T17:50:38.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.293+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=GI5B06Q9ZRRR tag=237XLZ5B2XJ4 type=write size=1897286 2024-03-04T17:50:38.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.297+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=BPKAYKJCCXKS type=del size=3238921 2024-03-04T17:50:38.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.301+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=7CTMCFWVW813 type=del 2024-03-04T17:50:38.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.305+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~864Z8VH9X22C.0 size=1048576 2024-03-04T17:50:38.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.309+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~864Z8VH9X22C.1 size=1048576 2024-03-04T17:50:38.310 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.313+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~864Z8VH9X22C.2 size=3081 2024-03-04T17:50:38.310 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.313+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=864Z8VH9X22C tag=00TOY62DHSJ7 type=write size=3148809 2024-03-04T17:50:38.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.317+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=KHL46VPFUGGP type=del 2024-03-04T17:50:38.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.321+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=9T90DZUV7R2V type=del size=3523698 2024-03-04T17:50:38.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.329+0000 7fda08af98c0 1 < suggested remove operation key=XG2XYBCMUNY5 tag=VEB44VAK6Q1L type=del 2024-03-04T17:50:38.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.333+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=343KFT01PLGC type=del size=3489182 2024-03-04T17:50:38.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.341+0000 7fda08af98c0 1 < suggested update operation key=VJVORUG95FLC tag=XCVBADTB82KH type=del size=255483 2024-03-04T17:50:38.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.345+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~WELNVOK472RJ.0 size=1048576 2024-03-04T17:50:38.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.349+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~WELNVOK472RJ.1 size=1048576 2024-03-04T17:50:38.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.353+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~WELNVOK472RJ.2 size=771776 2024-03-04T17:50:38.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.353+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=WELNVOK472RJ tag=XKCOVW5E1ZE1 type=write size=3917504 2024-03-04T17:50:38.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.361+0000 7fda08af98c0 1 < suggested update operation key=M5BMX7B7RZYL tag=7Y2P3139WUR5 type=del size=2678877 2024-03-04T17:50:38.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.369+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~VVE6X1ZZ2GSY.0 size=1048576 2024-03-04T17:50:38.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.369+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~VVE6X1ZZ2GSY.1 size=1028909 2024-03-04T17:50:38.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.369+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=VVE6X1ZZ2GSY tag=V55XL0WFCKNS type=write size=3126061 2024-03-04T17:50:38.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.373+0000 7fda08af98c0 1 < completed write operation key=C1GK00FM2ZYA tag=I30OJZMGZ5XF type=write size=131186 2024-03-04T17:50:38.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.377+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=HR89Q48V7XAW type=del size=2575396 2024-03-04T17:50:38.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.381+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=7WNKDTO67BM3 type=del 2024-03-04T17:50:38.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.389+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~IBSH910DBNAN.0 size=87491 2024-03-04T17:50:38.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.389+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=IBSH910DBNAN tag=C4TI8JIR2SA2 type=write size=1136067 2024-03-04T17:50:38.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.393+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~YI9MW1S6JFDQ.0 size=224582 2024-03-04T17:50:38.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.401+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=YI9MW1S6JFDQ tag=IWOLD3JEIHZ5 type=write size=1273158 2024-03-04T17:50:38.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.409+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~VIROFTIX3Z0T.0 size=71786 2024-03-04T17:50:38.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.409+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=VIROFTIX3Z0T tag=115ZIK94XYCL type=write size=1120362 2024-03-04T17:50:38.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.413+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~KLP5ZRHM41CE.0 size=801796 2024-03-04T17:50:38.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.417+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=KLP5ZRHM41CE tag=4N2PD708W4MF type=write size=1850372 2024-03-04T17:50:38.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.421+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=BVXZ2VOCH81H type=write size=1019614 2024-03-04T17:50:38.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.429+0000 7fda08af98c0 1 < completed write operation key=2PA8JDZUOLO5 tag=M8C3H2G2MWDA type=write size=41214 2024-03-04T17:50:38.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.433+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=BYZCISI6CKLH type=del size=1736930 2024-03-04T17:50:38.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.437+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=8LJTU3SVF2G2 type=del 2024-03-04T17:50:38.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.445+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~BQZ3EAJA9EWZ.0 size=1048576 2024-03-04T17:50:38.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.449+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~BQZ3EAJA9EWZ.1 size=1048576 2024-03-04T17:50:38.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.453+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~BQZ3EAJA9EWZ.2 size=1030713 2024-03-04T17:50:38.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.453+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=BQZ3EAJA9EWZ tag=1RGOU5IIRMI1 type=write size=4176441 2024-03-04T17:50:38.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.453+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=K97EI2S4J3X3 type=del 2024-03-04T17:50:38.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.457+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=SLYOYDNEXGTY type=del size=4108907 2024-03-04T17:50:38.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.465+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=JSZ2111BXA81 type=del 2024-03-04T17:50:38.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.465+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=S32U30IHRHE1 type=del size=4083171 2024-03-04T17:50:38.469 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.469+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=RT0PNVVJ9V0J type=del 2024-03-04T17:50:38.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.477+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~65G6WYW4XBIT.0 size=1048576 2024-03-04T17:50:38.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.481+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~65G6WYW4XBIT.1 size=1048576 2024-03-04T17:50:38.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.485+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~65G6WYW4XBIT.2 size=750045 2024-03-04T17:50:38.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.485+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=65G6WYW4XBIT tag=E0IRHYSN65GB type=write size=3895773 2024-03-04T17:50:38.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.493+0000 7fda08af98c0 1 < suggested remove operation key=HCA7QLGM2O4O tag=O54JT7JZGZJR type=del 2024-03-04T17:50:38.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.497+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=V1M1BDOIBOC4 type=del size=3730393 2024-03-04T17:50:38.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.501+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=BPKAYKJCCXKS type=del 2024-03-04T17:50:38.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.505+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=2CGRCMTJ1ZRU type=del size=1650498 2024-03-04T17:50:38.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.509+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~864Z8VH9X22C.0 size=1048576 2024-03-04T17:50:38.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.513+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~864Z8VH9X22C.1 size=1048576 2024-03-04T17:50:38.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.513+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~864Z8VH9X22C.2 size=3081 2024-03-04T17:50:38.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.521+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=864Z8VH9X22C tag=00TOY62DHSJ7 type=write size=3148809 2024-03-04T17:50:38.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.525+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=7JRH43GK8E9Q type=del size=2384026 2024-03-04T17:50:38.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.529+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=9T90DZUV7R2V type=del 2024-03-04T17:50:38.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.533+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=P5DB1A267PZZ type=del size=2074301 2024-03-04T17:50:38.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.537+0000 7fda08af98c0 -1 < failed to complete operation key=M5BMX7B7RZYL tag=343KFT01PLGC type=del size=3489182: (22) Invalid argument 2024-03-04T17:50:38.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.541+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~7UMFDJJC2XQF.0 size=1048576 2024-03-04T17:50:38.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.545+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~7UMFDJJC2XQF.1 size=287548 2024-03-04T17:50:38.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.545+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=7UMFDJJC2XQF tag=3993CE1DBQP5 type=write size=2384700 2024-03-04T17:50:38.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.545+0000 7fda08af98c0 1 < canceled part key=_multipart_8YL7C9PI58TK.2~WELNVOK472RJ.0 size=1048576 2024-03-04T17:50:38.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.545+0000 7fda08af98c0 1 < canceled part key=_multipart_8YL7C9PI58TK.2~WELNVOK472RJ.1 size=1048576 2024-03-04T17:50:38.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.545+0000 7fda08af98c0 1 < canceled part key=_multipart_8YL7C9PI58TK.2~WELNVOK472RJ.2 size=771776 2024-03-04T17:50:38.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.553+0000 7fda08af98c0 1 < canceled multipart upload key=8YL7C9PI58TK upload=WELNVOK472RJ tag=XKCOVW5E1ZE1 type=write size=3917504 2024-03-04T17:50:38.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.561+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~TOIRRVIAX9HR.0 size=1048576 2024-03-04T17:50:38.561 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.561+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~TOIRRVIAX9HR.1 size=125870 2024-03-04T17:50:38.561 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.561+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=TOIRRVIAX9HR tag=3BG6VF1OFQFS type=write size=2223022 2024-03-04T17:50:38.565 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.565+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~VVE6X1ZZ2GSY.0 size=1048576 2024-03-04T17:50:38.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.569+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~VVE6X1ZZ2GSY.1 size=1028909 2024-03-04T17:50:38.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.577+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=VVE6X1ZZ2GSY tag=V55XL0WFCKNS type=write size=3126061 2024-03-04T17:50:38.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.581+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=350UTEK72IXR type=del size=3435720 2024-03-04T17:50:38.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.585+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=HR89Q48V7XAW type=del 2024-03-04T17:50:38.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.589+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=83V6LAMXZCIR type=del size=3889047 2024-03-04T17:50:38.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.593+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~IBSH910DBNAN.0 size=87491 2024-03-04T17:50:38.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.601+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=IBSH910DBNAN tag=C4TI8JIR2SA2 type=write size=1136067 2024-03-04T17:50:38.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.605+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=6ZM3LZ2EY4ET type=del size=3292519 2024-03-04T17:50:38.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.609+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~VIROFTIX3Z0T.0 size=71786 2024-03-04T17:50:38.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.613+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=VIROFTIX3Z0T tag=115ZIK94XYCL type=write size=1120362 2024-03-04T17:50:38.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.621+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~9J33S37BLDA6.0 size=1048576 2024-03-04T17:50:38.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.625+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~9J33S37BLDA6.1 size=276114 2024-03-04T17:50:38.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.625+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=9J33S37BLDA6 tag=L5X1JF7VAQ6Y type=write size=2373266 2024-03-04T17:50:38.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.625+0000 7fda08af98c0 1 < completed write operation key=0IGSMFORCSL9 tag=BVXZ2VOCH81H type=write size=1019614 2024-03-04T17:50:38.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.629+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=2CF5Z68TP5SP type=del size=626597 2024-03-04T17:50:38.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.633+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=BYZCISI6CKLH type=del 2024-03-04T17:50:38.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.637+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=VGKXQH86SRVZ type=del size=3531562 2024-03-04T17:50:38.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.641+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~BQZ3EAJA9EWZ.0 size=1048576 2024-03-04T17:50:38.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.645+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~BQZ3EAJA9EWZ.1 size=1048576 2024-03-04T17:50:38.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.649+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~BQZ3EAJA9EWZ.2 size=1030713 2024-03-04T17:50:38.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.653+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=BQZ3EAJA9EWZ tag=1RGOU5IIRMI1 type=write size=4176441 2024-03-04T17:50:38.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.657+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=J7AFLMG3LWEY type=del size=2537425 2024-03-04T17:50:38.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.661+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=SLYOYDNEXGTY type=del 2024-03-04T17:50:38.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.665+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=3LD2KK9IRWHQ type=write size=220497 2024-03-04T17:50:38.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.669+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=S32U30IHRHE1 type=del 2024-03-04T17:50:38.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.677+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~JSWKIV52C0PJ.0 size=1048576 2024-03-04T17:50:38.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.681+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~JSWKIV52C0PJ.1 size=813084 2024-03-04T17:50:38.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.681+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=JSWKIV52C0PJ tag=K0WIOTB99707 type=write size=2910236 2024-03-04T17:50:38.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.685+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~65G6WYW4XBIT.0 size=1048576 2024-03-04T17:50:38.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.685+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~65G6WYW4XBIT.1 size=1048576 2024-03-04T17:50:38.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.689+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~65G6WYW4XBIT.2 size=750045 2024-03-04T17:50:38.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.697+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=65G6WYW4XBIT tag=E0IRHYSN65GB type=write size=3895773 2024-03-04T17:50:38.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.701+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=5ITU4SVH4S4Z type=del size=164719 2024-03-04T17:50:38.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.701+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=V1M1BDOIBOC4 type=del 2024-03-04T17:50:38.705 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.705+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=5E0QPD41ZZFF type=del size=2116514 2024-03-04T17:50:38.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.709+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=2CGRCMTJ1ZRU type=del 2024-03-04T17:50:38.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.713+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=9WGIB8XLCF2L type=del size=3687104 2024-03-04T17:50:38.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.717+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=7JRH43GK8E9Q type=del 2024-03-04T17:50:38.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.725+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8LAPNA24QZQT.0 size=1048576 2024-03-04T17:50:38.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.729+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8LAPNA24QZQT.1 size=356973 2024-03-04T17:50:38.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.729+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=8LAPNA24QZQT tag=3VDVDYJYZ2M0 type=write size=2454125 2024-03-04T17:50:38.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.733+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=P5DB1A267PZZ type=del 2024-03-04T17:50:38.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.737+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=C1SW1WMQXUN4 type=write size=611004 2024-03-04T17:50:38.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.741+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~7UMFDJJC2XQF.0 size=1048576 2024-03-04T17:50:38.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.741+0000 7fda08af98c0 1 < canceled part key=_multipart_FNWA2N9U14HO.2~7UMFDJJC2XQF.1 size=287548 2024-03-04T17:50:38.745 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.745+0000 7fda08af98c0 1 < canceled multipart upload key=FNWA2N9U14HO upload=7UMFDJJC2XQF tag=3993CE1DBQP5 type=write size=2384700 2024-03-04T17:50:38.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.749+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=D7QNECFAH7LB type=write size=117990 2024-03-04T17:50:38.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.753+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~TOIRRVIAX9HR.0 size=1048576 2024-03-04T17:50:38.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.757+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~TOIRRVIAX9HR.1 size=125870 2024-03-04T17:50:38.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.765+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=TOIRRVIAX9HR tag=3BG6VF1OFQFS type=write size=2223022 2024-03-04T17:50:38.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.769+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=YWQKI1Z8ZD79 type=del size=929660 2024-03-04T17:50:38.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.773+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=350UTEK72IXR type=del 2024-03-04T17:50:38.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.777+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=K4T9FPPSJTRH type=write size=390416 2024-03-04T17:50:38.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.781+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=83V6LAMXZCIR type=del 2024-03-04T17:50:38.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.785+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=S3OFTX6AUUON type=write size=995328 2024-03-04T17:50:38.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.789+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=6ZM3LZ2EY4ET type=del 2024-03-04T17:50:38.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.797+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=BODKPRJD8O93 type=del size=1186530 2024-03-04T17:50:38.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.797+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~9J33S37BLDA6.0 size=1048576 2024-03-04T17:50:38.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.801+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~9J33S37BLDA6.1 size=276114 2024-03-04T17:50:38.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.809+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=9J33S37BLDA6 tag=L5X1JF7VAQ6Y type=write size=2373266 2024-03-04T17:50:38.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.817+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~0B4EKGWYO0YM.0 size=1048576 2024-03-04T17:50:38.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.821+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~0B4EKGWYO0YM.1 size=421069 2024-03-04T17:50:38.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.821+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=0B4EKGWYO0YM tag=PGJRJ12N3G0L type=write size=2518221 2024-03-04T17:50:38.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.821+0000 7fda08af98c0 1 < canceled operation key=G0BHMSCO0XLF tag=2CF5Z68TP5SP type=del size=626597 2024-03-04T17:50:38.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.825+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=UTXD2NKO1H97 type=write size=377171 2024-03-04T17:50:38.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.829+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=VGKXQH86SRVZ type=del 2024-03-04T17:50:38.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.833+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=M9QSCT7O1174 type=del size=760924 2024-03-04T17:50:38.840 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.841+0000 7fda08af98c0 1 < suggested update operation key=VJVORUG95FLC tag=J7AFLMG3LWEY type=del size=2537425 2024-03-04T17:50:38.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.845+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=867ESU9ZBHG0 type=del size=3212084 2024-03-04T17:50:38.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.853+0000 7fda08af98c0 1 < canceled operation key=750NSVFBS03A tag=3LD2KK9IRWHQ type=write size=220497 2024-03-04T17:50:38.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.857+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~D2OXM6O9X512.0 size=809965 2024-03-04T17:50:38.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.857+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=D2OXM6O9X512 tag=DBLEPPRIMD6B type=write size=1858541 2024-03-04T17:50:38.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.861+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~JSWKIV52C0PJ.0 size=1048576 2024-03-04T17:50:38.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.865+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~JSWKIV52C0PJ.1 size=813084 2024-03-04T17:50:38.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.873+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=JSWKIV52C0PJ tag=K0WIOTB99707 type=write size=2910236 2024-03-04T17:50:38.875 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.877+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=S6UBXD8PNU4J type=del size=296294 2024-03-04T17:50:38.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.881+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=5ITU4SVH4S4Z type=del 2024-03-04T17:50:38.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.885+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=C5XMMC9H2ECE type=del size=642906 2024-03-04T17:50:38.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.893+0000 7fda08af98c0 1 < suggested update operation key=09KBCA2XURTF tag=5E0QPD41ZZFF type=del size=2116514 2024-03-04T17:50:38.896 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.897+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=K8AHDR4Q6H6B type=del size=2979567 2024-03-04T17:50:38.903 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.905+0000 7fda08af98c0 1 < suggested update operation key=XG2XYBCMUNY5 tag=9WGIB8XLCF2L type=del size=3687104 2024-03-04T17:50:38.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.909+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=1Y6EGJQJ3FF2 type=write size=248670 2024-03-04T17:50:38.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.913+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~8LAPNA24QZQT.0 size=1048576 2024-03-04T17:50:38.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.917+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~8LAPNA24QZQT.1 size=356973 2024-03-04T17:50:38.920 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.925+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=8LAPNA24QZQT tag=3VDVDYJYZ2M0 type=write size=2454125 2024-03-04T17:50:38.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.925+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=Q8K67MM07GR7 type=del size=1310813 2024-03-04T17:50:38.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.933+0000 7fda08af98c0 1 < suggested remove operation key=Z2JQBN84QQIB tag=C1SW1WMQXUN4 type=write 2024-03-04T17:50:38.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.937+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=2DC9GEB6EE5U type=del size=2303528 2024-03-04T17:50:38.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.941+0000 7fda08af98c0 1 < completed write operation key=GDGTJZ21TG32 tag=D7QNECFAH7LB type=write size=117990 2024-03-04T17:50:38.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.949+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~9ALYFGSPB42W.0 size=1048576 2024-03-04T17:50:38.950 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.953+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~9ALYFGSPB42W.1 size=1048576 2024-03-04T17:50:38.953 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.953+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~9ALYFGSPB42W.2 size=718724 2024-03-04T17:50:38.953 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.953+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=9ALYFGSPB42W tag=49BEA5V8XR9W type=write size=3864452 2024-03-04T17:50:38.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.957+0000 7fda08af98c0 -1 < failed to complete operation key=09KBCA2XURTF tag=YWQKI1Z8ZD79 type=del size=929660: (22) Invalid argument 2024-03-04T17:50:38.964 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.965+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~Y4QEO4PXRNH0.0 size=595600 2024-03-04T17:50:38.964 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.965+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=Y4QEO4PXRNH0 tag=TR0VC2FJP4G4 type=write size=1644176 2024-03-04T17:50:38.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.969+0000 7fda08af98c0 1 < completed write operation key=FNWA2N9U14HO tag=K4T9FPPSJTRH type=write size=390416 2024-03-04T17:50:38.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.977+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~C24FQXWVOHXQ.0 size=992731 2024-03-04T17:50:38.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.977+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=C24FQXWVOHXQ tag=9IO5QCLYCBWJ type=write size=2041307 2024-03-04T17:50:38.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.981+0000 7fda08af98c0 -1 < failed to complete operation key=XG2XYBCMUNY5 tag=S3OFTX6AUUON type=write size=995328: (22) Invalid argument 2024-03-04T17:50:38.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.985+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=2S9KWVI0X8DE type=write size=369746 2024-03-04T17:50:38.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.989+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=BODKPRJD8O93 type=del 2024-03-04T17:50:38.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.997+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~V3RGPWMV6V1D.0 size=1048576 2024-03-04T17:50:38.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.997+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~V3RGPWMV6V1D.1 size=436494 2024-03-04T17:50:38.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:38.997+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=V3RGPWMV6V1D tag=YE4W5GZAZZBF type=write size=2533646 2024-03-04T17:50:39.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.001+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~0B4EKGWYO0YM.0 size=1048576 2024-03-04T17:50:39.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.005+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~0B4EKGWYO0YM.1 size=421069 2024-03-04T17:50:39.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.013+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=0B4EKGWYO0YM tag=PGJRJ12N3G0L type=write size=2518221 2024-03-04T17:50:39.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.017+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=VQ927W2KQGD1 type=del size=11248 2024-03-04T17:50:39.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.021+0000 7fda08af98c0 1 < completed write operation key=V1LDB2HXZ7EF tag=UTXD2NKO1H97 type=write size=377171 2024-03-04T17:50:39.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.025+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~9A08SHUMF13K.0 size=1048576 2024-03-04T17:50:39.027 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.029+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~9A08SHUMF13K.1 size=440125 2024-03-04T17:50:39.027 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.029+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=9A08SHUMF13K tag=6GL8TC553YGS type=write size=2537277 2024-03-04T17:50:39.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.037+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=M9QSCT7O1174 type=del 2024-03-04T17:50:39.040 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.041+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=IATFGH7YT2F3 type=del size=1543174 2024-03-04T17:50:39.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.045+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=867ESU9ZBHG0 type=del 2024-03-04T17:50:39.051 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.053+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~116UR9XJEYVV.0 size=1048576 2024-03-04T17:50:39.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.057+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~116UR9XJEYVV.1 size=1048576 2024-03-04T17:50:39.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.057+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~116UR9XJEYVV.2 size=633355 2024-03-04T17:50:39.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.057+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=116UR9XJEYVV tag=3CA0OGHUKOTR type=write size=3779083 2024-03-04T17:50:39.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.065+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~D2OXM6O9X512.0 size=809965 2024-03-04T17:50:39.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.069+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=D2OXM6O9X512 tag=DBLEPPRIMD6B type=write size=1858541 2024-03-04T17:50:39.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.073+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=QT3UHPAISMZX type=del size=3825239 2024-03-04T17:50:39.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.081+0000 7fda08af98c0 1 < suggested update operation key=09KBCA2XURTF tag=S6UBXD8PNU4J type=del size=296294 2024-03-04T17:50:39.084 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.085+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=3AIC7ECJ1OPO type=del size=518247 2024-03-04T17:50:39.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.089+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=C5XMMC9H2ECE type=del 2024-03-04T17:50:39.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.097+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~6HGG1U9ZD61B.0 size=1048576 2024-03-04T17:50:39.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.097+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~6HGG1U9ZD61B.1 size=1048576 2024-03-04T17:50:39.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.101+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~6HGG1U9ZD61B.2 size=752742 2024-03-04T17:50:39.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.101+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=6HGG1U9ZD61B tag=IOXC074C804L type=write size=3898470 2024-03-04T17:50:39.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.105+0000 7fda08af98c0 1 < canceled operation key=750NSVFBS03A tag=K8AHDR4Q6H6B type=del size=2979567 2024-03-04T17:50:39.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.109+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=QD0J9YNFFBND type=del size=1689563 2024-03-04T17:50:39.111 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.113+0000 7fda08af98c0 1 < canceled operation key=1OZKPHMPV3OQ tag=1Y6EGJQJ3FF2 type=write size=248670 2024-03-04T17:50:39.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.117+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=B78QAVVCS280 type=del size=3390415 2024-03-04T17:50:39.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.125+0000 7fda08af98c0 1 < suggested update operation key=G0BHMSCO0XLF tag=Q8K67MM07GR7 type=del size=1310813 2024-03-04T17:50:39.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.129+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=MNPSK9FZGN4M type=del size=60941 2024-03-04T17:50:39.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.133+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=2DC9GEB6EE5U type=del 2024-03-04T17:50:39.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.137+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=N80C93MPVWU2 type=del size=494088 2024-03-04T17:50:39.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.141+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~9ALYFGSPB42W.0 size=1048576 2024-03-04T17:50:39.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.145+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~9ALYFGSPB42W.1 size=1048576 2024-03-04T17:50:39.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.149+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~9ALYFGSPB42W.2 size=718724 2024-03-04T17:50:39.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.153+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=9ALYFGSPB42W tag=49BEA5V8XR9W type=write size=3864452 2024-03-04T17:50:39.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.157+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=8TQPUOR3NELJ type=del size=1970338 2024-03-04T17:50:39.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.161+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~Y4QEO4PXRNH0.0 size=595600 2024-03-04T17:50:39.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.165+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=Y4QEO4PXRNH0 tag=TR0VC2FJP4G4 type=write size=1644176 2024-03-04T17:50:39.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.173+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=56VQ1ZXV9CLF type=del size=3126327 2024-03-04T17:50:39.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.173+0000 7fda08af98c0 1 < canceled part key=_multipart_BYQFZSJ3WH07.2~C24FQXWVOHXQ.0 size=992731 2024-03-04T17:50:39.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.181+0000 7fda08af98c0 1 < canceled multipart upload key=BYQFZSJ3WH07 upload=C24FQXWVOHXQ tag=9IO5QCLYCBWJ type=write size=2041307 2024-03-04T17:50:39.183 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.185+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=1H0KW8P7IRCG type=del size=3356995 2024-03-04T17:50:39.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.189+0000 7fda08af98c0 1 < completed write operation key=FNWA2N9U14HO tag=2S9KWVI0X8DE type=write size=369746 2024-03-04T17:50:39.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.193+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=XQNVIRP27YBA type=write size=456116 2024-03-04T17:50:39.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.197+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~V3RGPWMV6V1D.0 size=1048576 2024-03-04T17:50:39.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.201+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~V3RGPWMV6V1D.1 size=436494 2024-03-04T17:50:39.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.205+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=V3RGPWMV6V1D tag=YE4W5GZAZZBF type=write size=2533646 2024-03-04T17:50:39.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.213+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~UOB2PN1A533V.0 size=1048576 2024-03-04T17:50:39.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.217+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~UOB2PN1A533V.1 size=1048576 2024-03-04T17:50:39.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.221+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~UOB2PN1A533V.2 size=345172 2024-03-04T17:50:39.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.221+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=UOB2PN1A533V tag=40LSWONOOYKL type=write size=3490900 2024-03-04T17:50:39.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.225+0000 7fda08af98c0 1 < canceled operation key=XG5I3SDRE5JM tag=VQ927W2KQGD1 type=del size=11248 2024-03-04T17:50:39.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.229+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=6T5X7E56F4ZX type=del size=3361294 2024-03-04T17:50:39.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.233+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~9A08SHUMF13K.0 size=1048576 2024-03-04T17:50:39.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.237+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~9A08SHUMF13K.1 size=440125 2024-03-04T17:50:39.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.241+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=9A08SHUMF13K tag=6GL8TC553YGS type=write size=2537277 2024-03-04T17:50:39.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.249+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~36ALNNYW1YWO.0 size=211841 2024-03-04T17:50:39.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.249+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=36ALNNYW1YWO tag=3C3130TVE01S type=write size=1260417 2024-03-04T17:50:39.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.253+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=IATFGH7YT2F3 type=del 2024-03-04T17:50:39.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.261+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~6SDIETMGSODV.0 size=357992 2024-03-04T17:50:39.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.261+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=6SDIETMGSODV tag=ZACNT50TX0PB type=write size=1406568 2024-03-04T17:50:39.262 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.265+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~116UR9XJEYVV.0 size=1048576 2024-03-04T17:50:39.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.265+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~116UR9XJEYVV.1 size=1048576 2024-03-04T17:50:39.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.269+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~116UR9XJEYVV.2 size=633355 2024-03-04T17:50:39.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.277+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=116UR9XJEYVV tag=3CA0OGHUKOTR type=write size=3779083 2024-03-04T17:50:39.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.281+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=4J7ICS3LEZXR type=del size=3397194 2024-03-04T17:50:39.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.285+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=QT3UHPAISMZX type=del 2024-03-04T17:50:39.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.289+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=SLZ7QWS75S3X type=del size=3432261 2024-03-04T17:50:39.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.293+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=3AIC7ECJ1OPO type=del 2024-03-04T17:50:39.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.301+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~6NS31IDC63VB.0 size=1048576 2024-03-04T17:50:39.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.301+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~6NS31IDC63VB.1 size=1048576 2024-03-04T17:50:39.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.305+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~6NS31IDC63VB.2 size=448476 2024-03-04T17:50:39.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.305+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=6NS31IDC63VB tag=JW8FQC0PFCFO type=write size=3594204 2024-03-04T17:50:39.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.309+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~6HGG1U9ZD61B.0 size=1048576 2024-03-04T17:50:39.312 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.313+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~6HGG1U9ZD61B.1 size=1048576 2024-03-04T17:50:39.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.317+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~6HGG1U9ZD61B.2 size=752742 2024-03-04T17:50:39.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.321+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=6HGG1U9ZD61B tag=IOXC074C804L type=write size=3898470 2024-03-04T17:50:39.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.325+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=90Y43R4TFFLS type=del size=3503206 2024-03-04T17:50:39.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.333+0000 7fda08af98c0 1 < suggested remove operation key=XG2XYBCMUNY5 tag=QD0J9YNFFBND type=del 2024-03-04T17:50:39.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.337+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=SFD3IJ2I0UKN type=del size=714865 2024-03-04T17:50:39.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.341+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=B78QAVVCS280 type=del 2024-03-04T17:50:39.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.345+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=GNINU4MVT8XR type=del size=3337031 2024-03-04T17:50:39.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.349+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=MNPSK9FZGN4M type=del 2024-03-04T17:50:39.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.353+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=2LXJZ4UX1RCD type=write size=304385 2024-03-04T17:50:39.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.357+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=N80C93MPVWU2 type=del 2024-03-04T17:50:39.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.361+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=FC4VZBO8BQ3H type=write size=928156 2024-03-04T17:50:39.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.365+0000 7fda08af98c0 -1 < failed to complete operation key=XG2XYBCMUNY5 tag=8TQPUOR3NELJ type=del size=1970338: (22) Invalid argument 2024-03-04T17:50:39.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.369+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=YDEQ4WUU5R5K type=del size=3444624 2024-03-04T17:50:39.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.373+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=56VQ1ZXV9CLF type=del 2024-03-04T17:50:39.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.381+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~M4J1T8JTN7V7.0 size=1048576 2024-03-04T17:50:39.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.381+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~M4J1T8JTN7V7.1 size=1048576 2024-03-04T17:50:39.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.385+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~M4J1T8JTN7V7.2 size=469535 2024-03-04T17:50:39.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.385+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=M4J1T8JTN7V7 tag=40ENGKR6LVZL type=write size=3615263 2024-03-04T17:50:39.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.389+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=1H0KW8P7IRCG type=del 2024-03-04T17:50:39.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.393+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=K3Y5ZMP3MF9E type=del size=4037245 2024-03-04T17:50:39.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.397+0000 7fda08af98c0 1 < canceled operation key=HCKJZUMAK0Z0 tag=XQNVIRP27YBA type=write size=456116 2024-03-04T17:50:39.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.401+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=1VDW01OOCP9Y type=write size=568601 2024-03-04T17:50:39.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.405+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~UOB2PN1A533V.0 size=1048576 2024-03-04T17:50:39.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.409+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~UOB2PN1A533V.1 size=1048576 2024-03-04T17:50:39.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.409+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~UOB2PN1A533V.2 size=345172 2024-03-04T17:50:39.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.417+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=UOB2PN1A533V tag=40LSWONOOYKL type=write size=3490900 2024-03-04T17:50:39.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.421+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=62LKLKFWWOP2 type=write size=808797 2024-03-04T17:50:39.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.425+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=6T5X7E56F4ZX type=del 2024-03-04T17:50:39.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.429+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=PYDSJ0MX0H1T type=del size=71998 2024-03-04T17:50:39.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.433+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~36ALNNYW1YWO.0 size=211841 2024-03-04T17:50:39.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.437+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=36ALNNYW1YWO tag=3C3130TVE01S type=write size=1260417 2024-03-04T17:50:39.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.441+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=GKLRKPR035F8 type=del size=2729615 2024-03-04T17:50:39.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.449+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~6SDIETMGSODV.0 size=357992 2024-03-04T17:50:39.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.453+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=6SDIETMGSODV tag=ZACNT50TX0PB type=write size=1406568 2024-03-04T17:50:39.455 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.457+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=RNTEAWKLS6VE type=write size=260359 2024-03-04T17:50:39.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.465+0000 7fda08af98c0 1 < suggested remove operation key=3UWSL7NJLACY tag=4J7ICS3LEZXR type=del 2024-03-04T17:50:39.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.473+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~KG04REFWDAMI.0 size=715332 2024-03-04T17:50:39.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.473+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=KG04REFWDAMI tag=9GT1NFOWGMMK type=write size=1763908 2024-03-04T17:50:39.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.477+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=SLZ7QWS75S3X type=del 2024-03-04T17:50:39.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.481+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~HWWUVFE3GA9Y.0 size=893220 2024-03-04T17:50:39.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.481+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=HWWUVFE3GA9Y tag=M1735SICTZSI type=write size=1941796 2024-03-04T17:50:39.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.485+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~6NS31IDC63VB.0 size=1048576 2024-03-04T17:50:39.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.489+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~6NS31IDC63VB.1 size=1048576 2024-03-04T17:50:39.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.493+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~6NS31IDC63VB.2 size=448476 2024-03-04T17:50:39.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.497+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=6NS31IDC63VB tag=JW8FQC0PFCFO type=write size=3594204 2024-03-04T17:50:39.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.505+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~ZCL33TRQGDYK.0 size=1048576 2024-03-04T17:50:39.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.509+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~ZCL33TRQGDYK.1 size=282440 2024-03-04T17:50:39.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.509+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=ZCL33TRQGDYK tag=T1DNBFX9UZVD type=write size=2379592 2024-03-04T17:50:39.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.513+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=90Y43R4TFFLS type=del 2024-03-04T17:50:39.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.521+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~WARJAG6N4LSL.0 size=1048576 2024-03-04T17:50:39.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.521+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~WARJAG6N4LSL.1 size=1048576 2024-03-04T17:50:39.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.525+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~WARJAG6N4LSL.2 size=431466 2024-03-04T17:50:39.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.525+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=WARJAG6N4LSL tag=4BEWJ949XR45 type=write size=3577194 2024-03-04T17:50:39.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.537+0000 7fda08af98c0 1 < suggested update operation key=HCKJZUMAK0Z0 tag=SFD3IJ2I0UKN type=del size=714865 2024-03-04T17:50:39.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.541+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=LSI1KOT6TUFK type=write size=107855 2024-03-04T17:50:39.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.545+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=GNINU4MVT8XR type=del 2024-03-04T17:50:39.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.553+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~CK94YHN6K45Y.0 size=1048576 2024-03-04T17:50:39.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.553+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~CK94YHN6K45Y.1 size=312187 2024-03-04T17:50:39.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.553+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=CK94YHN6K45Y tag=703WBWTGXIMK type=write size=2409339 2024-03-04T17:50:39.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.561+0000 7fda08af98c0 1 < completed write operation key=0IGSMFORCSL9 tag=2LXJZ4UX1RCD type=write size=304385 2024-03-04T17:50:39.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.565+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=Y0ETL7R7TSH0 type=del size=738689 2024-03-04T17:50:39.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.573+0000 7fda08af98c0 1 < suggested remove operation key=G0BHMSCO0XLF tag=FC4VZBO8BQ3H type=write 2024-03-04T17:50:39.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.577+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=4H0AKO16O8MM type=del size=74374 2024-03-04T17:50:39.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.581+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=YDEQ4WUU5R5K type=del 2024-03-04T17:50:39.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.585+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=PSEVRCUPAYLI type=del size=1871 2024-03-04T17:50:39.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.589+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~M4J1T8JTN7V7.0 size=1048576 2024-03-04T17:50:39.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.593+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~M4J1T8JTN7V7.1 size=1048576 2024-03-04T17:50:39.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.597+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~M4J1T8JTN7V7.2 size=469535 2024-03-04T17:50:39.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.601+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=M4J1T8JTN7V7 tag=40ENGKR6LVZL type=write size=3615263 2024-03-04T17:50:39.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.609+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~2Y292SWE2PI9.0 size=15204 2024-03-04T17:50:39.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.609+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=2Y292SWE2PI9 tag=52RKMF0NDFIP type=write size=1063780 2024-03-04T17:50:39.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.613+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=K3Y5ZMP3MF9E type=del 2024-03-04T17:50:39.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.621+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~TF191106KFKZ.0 size=1048576 2024-03-04T17:50:39.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.625+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~TF191106KFKZ.1 size=460361 2024-03-04T17:50:39.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.625+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=TF191106KFKZ tag=UY45AHES9N7W type=write size=2557513 2024-03-04T17:50:39.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.633+0000 7fda08af98c0 1 < suggested update operation key=HCA7QLGM2O4O tag=1VDW01OOCP9Y type=write size=568601 2024-03-04T17:50:39.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.637+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=RUZA9O6RRP5Y type=write size=1023862 2024-03-04T17:50:39.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.641+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=62LKLKFWWOP2 type=write size=808797 2024-03-04T17:50:39.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.645+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=870HZ8GSAWV3 type=del size=373212 2024-03-04T17:50:39.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.649+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=PYDSJ0MX0H1T type=del 2024-03-04T17:50:39.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.653+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=XBV4V7KO6NL9 type=write size=257000 2024-03-04T17:50:39.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.657+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=GKLRKPR035F8 type=del 2024-03-04T17:50:39.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.661+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=5T3BNOYDKUBK type=write size=386053 2024-03-04T17:50:39.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.665+0000 7fda08af98c0 1 < completed write operation key=C1GK00FM2ZYA tag=RNTEAWKLS6VE type=write size=260359 2024-03-04T17:50:39.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.669+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~OIRDFVTA8LUT.0 size=1048576 2024-03-04T17:50:39.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.673+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~OIRDFVTA8LUT.1 size=1048576 2024-03-04T17:50:39.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.677+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~OIRDFVTA8LUT.2 size=426197 2024-03-04T17:50:39.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.677+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=OIRDFVTA8LUT tag=1JNUP7IBQB5L type=write size=3571925 2024-03-04T17:50:39.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.681+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~KG04REFWDAMI.0 size=715332 2024-03-04T17:50:39.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.685+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=KG04REFWDAMI tag=9GT1NFOWGMMK type=write size=1763908 2024-03-04T17:50:39.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.693+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~CDS09QMCV2W8.0 size=1048576 2024-03-04T17:50:39.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.697+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~CDS09QMCV2W8.1 size=1048576 2024-03-04T17:50:39.697 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.701+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~CDS09QMCV2W8.2 size=3397 2024-03-04T17:50:39.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.701+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=CDS09QMCV2W8 tag=PO79MOJI7P8H type=write size=3149125 2024-03-04T17:50:39.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.705+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~HWWUVFE3GA9Y.0 size=893220 2024-03-04T17:50:39.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.709+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=HWWUVFE3GA9Y tag=M1735SICTZSI type=write size=1941796 2024-03-04T17:50:39.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.717+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~1AAL6WRIVC3E.0 size=466244 2024-03-04T17:50:39.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.717+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=1AAL6WRIVC3E tag=020L2VNG5NMO type=write size=1514820 2024-03-04T17:50:39.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.721+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~ZCL33TRQGDYK.0 size=1048576 2024-03-04T17:50:39.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.721+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~ZCL33TRQGDYK.1 size=282440 2024-03-04T17:50:39.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.729+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=ZCL33TRQGDYK tag=T1DNBFX9UZVD type=write size=2379592 2024-03-04T17:50:39.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.733+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=CS0A8EZZHBNN type=del size=3920027 2024-03-04T17:50:39.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.737+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~WARJAG6N4LSL.0 size=1048576 2024-03-04T17:50:39.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.737+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~WARJAG6N4LSL.1 size=1048576 2024-03-04T17:50:39.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.741+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~WARJAG6N4LSL.2 size=431466 2024-03-04T17:50:39.745 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.745+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=WARJAG6N4LSL tag=4BEWJ949XR45 type=write size=3577194 2024-03-04T17:50:39.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.749+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=GKTZ07D9M12S type=del size=1535759 2024-03-04T17:50:39.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.753+0000 7fda08af98c0 1 < completed write operation key=XG2XYBCMUNY5 tag=LSI1KOT6TUFK type=write size=107855 2024-03-04T17:50:39.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.761+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~1S9BRTR6J5YJ.0 size=1048576 2024-03-04T17:50:39.762 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.765+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~1S9BRTR6J5YJ.1 size=876755 2024-03-04T17:50:39.762 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.765+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=1S9BRTR6J5YJ tag=4Y823L9G72O6 type=write size=2973907 2024-03-04T17:50:39.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.769+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~CK94YHN6K45Y.0 size=1048576 2024-03-04T17:50:39.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.769+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~CK94YHN6K45Y.1 size=312187 2024-03-04T17:50:39.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.777+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=CK94YHN6K45Y tag=703WBWTGXIMK type=write size=2409339 2024-03-04T17:50:39.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.781+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=M1CEETBA64IY type=del size=1785173 2024-03-04T17:50:39.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.781+0000 7fda08af98c0 -1 < failed to complete operation key=HCA7QLGM2O4O tag=Y0ETL7R7TSH0 type=del size=738689: (22) Invalid argument 2024-03-04T17:50:39.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.785+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=QSFCL8UEMFR9 type=write size=696582 2024-03-04T17:50:39.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.789+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=4H0AKO16O8MM type=del 2024-03-04T17:50:39.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.793+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=RSD95TBND2BN type=del size=2126413 2024-03-04T17:50:39.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.801+0000 7fda08af98c0 1 < suggested update operation key=DJCCQ6S0E7UG tag=PSEVRCUPAYLI type=del size=1871 2024-03-04T17:50:39.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.809+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~5E1M8B328UV0.0 size=1048576 2024-03-04T17:50:39.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.813+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~5E1M8B328UV0.1 size=13343 2024-03-04T17:50:39.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.813+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=5E1M8B328UV0 tag=YMDTDIW93WDK type=write size=2110495 2024-03-04T17:50:39.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.817+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~2Y292SWE2PI9.0 size=15204 2024-03-04T17:50:39.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.821+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=2Y292SWE2PI9 tag=52RKMF0NDFIP type=write size=1063780 2024-03-04T17:50:39.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.825+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=IH1SZNUA6AXO type=del size=2931170 2024-03-04T17:50:39.827 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.829+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~TF191106KFKZ.0 size=1048576 2024-03-04T17:50:39.830 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.833+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~TF191106KFKZ.1 size=460361 2024-03-04T17:50:39.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.837+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=TF191106KFKZ tag=UY45AHES9N7W type=write size=2557513 2024-03-04T17:50:39.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.841+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=IUU08VY110U2 type=del size=1346696 2024-03-04T17:50:39.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.845+0000 7fda08af98c0 1 < completed write operation key=X5MEJ7G8X9HE tag=RUZA9O6RRP5Y type=write size=1023862 2024-03-04T17:50:39.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.853+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~PLPXGU09LPTF.0 size=1048576 2024-03-04T17:50:39.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.853+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~PLPXGU09LPTF.1 size=1033300 2024-03-04T17:50:39.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.853+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=PLPXGU09LPTF tag=3IW0A28Q1F7M type=write size=3130452 2024-03-04T17:50:39.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.857+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=870HZ8GSAWV3 type=del 2024-03-04T17:50:39.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.861+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=LK9C1T7FP0U2 type=del size=2356259 2024-03-04T17:50:39.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.865+0000 7fda08af98c0 1 < completed write operation key=2PA8JDZUOLO5 tag=XBV4V7KO6NL9 type=write size=257000 2024-03-04T17:50:39.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.873+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~39HXSNDEHP99.0 size=1048576 2024-03-04T17:50:39.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.877+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~39HXSNDEHP99.1 size=1048576 2024-03-04T17:50:39.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.877+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~39HXSNDEHP99.2 size=540856 2024-03-04T17:50:39.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.877+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=39HXSNDEHP99 tag=4HDU0S1VKPZ0 type=write size=3686584 2024-03-04T17:50:39.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.885+0000 7fda08af98c0 1 < completed write operation key=Z2JQBN84QQIB tag=5T3BNOYDKUBK type=write size=386053 2024-03-04T17:50:39.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.889+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=GUI6V39VQO3G type=del size=2606953 2024-03-04T17:50:39.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.893+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~OIRDFVTA8LUT.0 size=1048576 2024-03-04T17:50:39.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.893+0000 7fda08af98c0 1 < canceled part key=_multipart_8YL7C9PI58TK.2~OIRDFVTA8LUT.1 size=1048576 2024-03-04T17:50:39.894 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.897+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~OIRDFVTA8LUT.2 size=426197 2024-03-04T17:50:39.900 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.901+0000 7fda08af98c0 1 < canceled multipart upload key=8YL7C9PI58TK upload=OIRDFVTA8LUT tag=1JNUP7IBQB5L type=write size=3571925 2024-03-04T17:50:39.903 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.905+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=K2B6IQC1NXAW type=del size=1146227 2024-03-04T17:50:39.904 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.905+0000 7fda08af98c0 1 < canceled part key=_multipart_L0FVKWD7EQRW.2~CDS09QMCV2W8.0 size=1048576 2024-03-04T17:50:39.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.909+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~CDS09QMCV2W8.1 size=1048576 2024-03-04T17:50:39.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.913+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~CDS09QMCV2W8.2 size=3397 2024-03-04T17:50:39.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.917+0000 7fda08af98c0 1 < canceled multipart upload key=L0FVKWD7EQRW upload=CDS09QMCV2W8 tag=PO79MOJI7P8H type=write size=3149125 2024-03-04T17:50:39.923 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.925+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=7ICUBKNCB3RA type=del size=2341367 2024-03-04T17:50:39.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.929+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~1AAL6WRIVC3E.0 size=466244 2024-03-04T17:50:39.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.933+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=1AAL6WRIVC3E tag=020L2VNG5NMO type=write size=1514820 2024-03-04T17:50:39.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.937+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=7FF6EKOVQEUG type=del size=310334 2024-03-04T17:50:39.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.949+0000 7fda08af98c0 1 < suggested update operation key=VJVORUG95FLC tag=CS0A8EZZHBNN type=del size=3920027 2024-03-04T17:50:39.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.953+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~732C9H3UAGKG.0 size=232809 2024-03-04T17:50:39.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.953+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=732C9H3UAGKG tag=0T0GCMUZVY8C type=write size=1281385 2024-03-04T17:50:39.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.957+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=GKTZ07D9M12S type=del 2024-03-04T17:50:39.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.965+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~6XSUD0JL9M20.0 size=1048576 2024-03-04T17:50:39.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.969+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~6XSUD0JL9M20.1 size=879656 2024-03-04T17:50:39.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.969+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=6XSUD0JL9M20 tag=AVLZKLWS4ZMU type=write size=2976808 2024-03-04T17:50:39.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.969+0000 7fda08af98c0 1 < canceled part key=_multipart_HCKJZUMAK0Z0.2~1S9BRTR6J5YJ.0 size=1048576 2024-03-04T17:50:39.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.973+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~1S9BRTR6J5YJ.1 size=876755 2024-03-04T17:50:39.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.977+0000 7fda08af98c0 1 < canceled multipart upload key=HCKJZUMAK0Z0 upload=1S9BRTR6J5YJ tag=4Y823L9G72O6 type=write size=2973907 2024-03-04T17:50:39.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.981+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=P0CFH9AOABJC type=write size=249400 2024-03-04T17:50:39.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.985+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=M1CEETBA64IY type=del 2024-03-04T17:50:39.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.993+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=AQF3HT88RXOK type=del size=4092149 2024-03-04T17:50:39.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:39.997+0000 7fda08af98c0 1 < completed write operation key=Z3J7W9V8HQKH tag=QSFCL8UEMFR9 type=write size=696582 2024-03-04T17:50:39.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.001+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=DB6534KFLDS5 type=del size=642605 2024-03-04T17:50:40.002 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.005+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=RSD95TBND2BN type=del 2024-03-04T17:50:40.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.009+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=3HLLX7YYU8VA type=del size=1780808 2024-03-04T17:50:40.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.013+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~5E1M8B328UV0.0 size=1048576 2024-03-04T17:50:40.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.013+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~5E1M8B328UV0.1 size=13343 2024-03-04T17:50:40.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.021+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=5E1M8B328UV0 tag=YMDTDIW93WDK type=write size=2110495 2024-03-04T17:50:40.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.025+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=MJ7HLN0QEWLI type=del size=3848954 2024-03-04T17:50:40.025 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.029+0000 7fda08af98c0 -1 < failed to complete operation key=VJVORUG95FLC tag=IH1SZNUA6AXO type=del size=2931170: (22) Invalid argument 2024-03-04T17:50:40.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.033+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=247IB6OVK57I type=del size=2062857 2024-03-04T17:50:40.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.033+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=IUU08VY110U2 type=del 2024-03-04T17:50:40.039 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.041+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~S0K5GMOU3BFT.0 size=1012541 2024-03-04T17:50:40.039 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.041+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=S0K5GMOU3BFT tag=KG3PZ9XUW195 type=write size=2061117 2024-03-04T17:50:40.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.045+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~PLPXGU09LPTF.0 size=1048576 2024-03-04T17:50:40.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.049+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~PLPXGU09LPTF.1 size=1033300 2024-03-04T17:50:40.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.053+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=PLPXGU09LPTF tag=3IW0A28Q1F7M type=write size=3130452 2024-03-04T17:50:40.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.057+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=3URHCZEZH4HY type=write size=397084 2024-03-04T17:50:40.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.061+0000 7fda08af98c0 1 < canceled operation key=Z2JQBN84QQIB tag=LK9C1T7FP0U2 type=del size=2356259 2024-03-04T17:50:40.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.065+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=7ABASX83B8ZD type=del size=2796675 2024-03-04T17:50:40.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.069+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~39HXSNDEHP99.0 size=1048576 2024-03-04T17:50:40.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.073+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~39HXSNDEHP99.1 size=1048576 2024-03-04T17:50:40.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.077+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~39HXSNDEHP99.2 size=540856 2024-03-04T17:50:40.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.081+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=39HXSNDEHP99 tag=4HDU0S1VKPZ0 type=write size=3686584 2024-03-04T17:50:40.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.089+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~QPB7SDV4S0Y6.0 size=467270 2024-03-04T17:50:40.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.089+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=QPB7SDV4S0Y6 tag=I4ZSB7SBP5Y7 type=write size=1515846 2024-03-04T17:50:40.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.097+0000 7fda08af98c0 1 < suggested remove operation key=750NSVFBS03A tag=GUI6V39VQO3G type=del 2024-03-04T17:50:40.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.101+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=I4PGPW0L0ZW8 type=del size=620597 2024-03-04T17:50:40.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.101+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=K2B6IQC1NXAW type=del 2024-03-04T17:50:40.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.109+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=MQWSXMR2OZD1 type=del size=1874459 2024-03-04T17:50:40.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.109+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=7ICUBKNCB3RA type=del 2024-03-04T17:50:40.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.117+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=JX175KATC1W9 type=write size=882270 2024-03-04T17:50:40.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.117+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=7FF6EKOVQEUG type=del 2024-03-04T17:50:40.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.125+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~831K8PULV4N1.0 size=857509 2024-03-04T17:50:40.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.125+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=831K8PULV4N1 tag=QGRTWJJUB4HE type=write size=1906085 2024-03-04T17:50:40.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.129+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~732C9H3UAGKG.0 size=232809 2024-03-04T17:50:40.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.137+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=732C9H3UAGKG tag=0T0GCMUZVY8C type=write size=1281385 2024-03-04T17:50:40.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.141+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=IKNAIACNR8CK type=write size=313732 2024-03-04T17:50:40.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.145+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~6XSUD0JL9M20.0 size=1048576 2024-03-04T17:50:40.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.145+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~6XSUD0JL9M20.1 size=879656 2024-03-04T17:50:40.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.153+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=6XSUD0JL9M20 tag=AVLZKLWS4ZMU type=write size=2976808 2024-03-04T17:50:40.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.157+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=GPRFTLHTJMX8 type=del size=3411574 2024-03-04T17:50:40.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.161+0000 7fda08af98c0 1 < completed write operation key=JA4JK99I0942 tag=P0CFH9AOABJC type=write size=249400 2024-03-04T17:50:40.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.165+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~WLQVD3QXKG8K.0 size=1048576 2024-03-04T17:50:40.168 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.169+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~WLQVD3QXKG8K.1 size=1048576 2024-03-04T17:50:40.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.173+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~WLQVD3QXKG8K.2 size=1034210 2024-03-04T17:50:40.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.173+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=WLQVD3QXKG8K tag=OJYPMKQWD1GR type=write size=4179938 2024-03-04T17:50:40.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.177+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=AQF3HT88RXOK type=del 2024-03-04T17:50:40.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.181+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=NF3HNRKK41IU type=del size=2797056 2024-03-04T17:50:40.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.189+0000 7fda08af98c0 1 < suggested update operation key=G0BHMSCO0XLF tag=DB6534KFLDS5 type=del size=642605 2024-03-04T17:50:40.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.197+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~WRVAF083WVB8.0 size=1048576 2024-03-04T17:50:40.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.197+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~WRVAF083WVB8.1 size=1048576 2024-03-04T17:50:40.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.201+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~WRVAF083WVB8.2 size=1026816 2024-03-04T17:50:40.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.201+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=WRVAF083WVB8 tag=AYLG7UZMB2VN type=write size=4172544 2024-03-04T17:50:40.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.209+0000 7fda08af98c0 1 < suggested update operation key=X5MEJ7G8X9HE tag=3HLLX7YYU8VA type=del size=1780808 2024-03-04T17:50:40.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.213+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=RP6PY6Z6S9PT type=del size=381348 2024-03-04T17:50:40.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.221+0000 7fda08af98c0 1 < suggested update operation key=ITIVF3UAT9NK tag=MJ7HLN0QEWLI type=del size=3848954 2024-03-04T17:50:40.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.229+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~G5M9OECTJ2II.0 size=1048576 2024-03-04T17:50:40.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.233+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~G5M9OECTJ2II.1 size=1048576 2024-03-04T17:50:40.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.233+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~G5M9OECTJ2II.2 size=77343 2024-03-04T17:50:40.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.233+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=G5M9OECTJ2II tag=WYK3961UEGMC type=write size=3223071 2024-03-04T17:50:40.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.237+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=247IB6OVK57I type=del 2024-03-04T17:50:40.245 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.245+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=1XTQG5CACXCR type=del size=1666059 2024-03-04T17:50:40.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.249+0000 7fda08af98c0 1 < canceled part key=_multipart_HCA7QLGM2O4O.2~S0K5GMOU3BFT.0 size=1012541 2024-03-04T17:50:40.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.253+0000 7fda08af98c0 1 < canceled multipart upload key=HCA7QLGM2O4O upload=S0K5GMOU3BFT tag=KG3PZ9XUW195 type=write size=2061117 2024-03-04T17:50:40.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.261+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~3YAVM1AV9UVB.0 size=621051 2024-03-04T17:50:40.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.261+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=3YAVM1AV9UVB tag=Y2I1EKSVJQG6 type=write size=1669627 2024-03-04T17:50:40.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.269+0000 7fda08af98c0 1 < suggested remove operation key=09KBCA2XURTF tag=3URHCZEZH4HY type=write 2024-03-04T17:50:40.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.277+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~CR9Q0LBXWZQ9.0 size=348987 2024-03-04T17:50:40.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.277+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=CR9Q0LBXWZQ9 tag=GVB3BYE8PUKI type=write size=1397563 2024-03-04T17:50:40.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.285+0000 7fda08af98c0 1 < suggested remove operation key=8YL7C9PI58TK tag=7ABASX83B8ZD type=del 2024-03-04T17:50:40.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.289+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~YBEH7G3RENCQ.0 size=1048576 2024-03-04T17:50:40.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.293+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~YBEH7G3RENCQ.1 size=1048576 2024-03-04T17:50:40.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.297+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~YBEH7G3RENCQ.2 size=135793 2024-03-04T17:50:40.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.297+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=YBEH7G3RENCQ tag=XP7SZ96GXY67 type=write size=3281521 2024-03-04T17:50:40.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.301+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~QPB7SDV4S0Y6.0 size=467270 2024-03-04T17:50:40.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.305+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=QPB7SDV4S0Y6 tag=I4ZSB7SBP5Y7 type=write size=1515846 2024-03-04T17:50:40.308 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.309+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=UO8PXWZE1VBJ type=del size=2353450 2024-03-04T17:50:40.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.313+0000 7fda08af98c0 1 < canceled operation key=BYQFZSJ3WH07 tag=I4PGPW0L0ZW8 type=del size=620597 2024-03-04T17:50:40.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.321+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~VA8KK6F63ZYY.0 size=1048576 2024-03-04T17:50:40.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.325+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~VA8KK6F63ZYY.1 size=1048576 2024-03-04T17:50:40.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.329+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~VA8KK6F63ZYY.2 size=754129 2024-03-04T17:50:40.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.329+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=VA8KK6F63ZYY tag=C5175AX626QM type=write size=3899857 2024-03-04T17:50:40.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.333+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=MQWSXMR2OZD1 type=del 2024-03-04T17:50:40.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.337+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=AQW4N2ISJP57 type=write size=58146 2024-03-04T17:50:40.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.341+0000 7fda08af98c0 -1 < failed to complete operation key=09KBCA2XURTF tag=JX175KATC1W9 type=write size=882270: (22) Invalid argument 2024-03-04T17:50:40.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.349+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~K8C3EAMZ12XU.0 size=1048576 2024-03-04T17:50:40.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.353+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~K8C3EAMZ12XU.1 size=1029188 2024-03-04T17:50:40.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.353+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=K8C3EAMZ12XU tag=8FU0TB3SOBWT type=write size=3126340 2024-03-04T17:50:40.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.353+0000 7fda08af98c0 1 < canceled part key=_multipart_TOFWDFH0HL3Q.2~831K8PULV4N1.0 size=857509 2024-03-04T17:50:40.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.357+0000 7fda08af98c0 1 < canceled multipart upload key=TOFWDFH0HL3Q upload=831K8PULV4N1 tag=QGRTWJJUB4HE type=write size=1906085 2024-03-04T17:50:40.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.361+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=HLMDGA010BNX type=del size=894602 2024-03-04T17:50:40.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.365+0000 7fda08af98c0 1 < completed write operation key=M5BMX7B7RZYL tag=IKNAIACNR8CK type=write size=313732 2024-03-04T17:50:40.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.373+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~LJVH2146EGL5.0 size=1048576 2024-03-04T17:50:40.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.377+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~LJVH2146EGL5.1 size=1048576 2024-03-04T17:50:40.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.377+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~LJVH2146EGL5.2 size=602889 2024-03-04T17:50:40.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.377+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=LJVH2146EGL5 tag=BYSLR3KGFR2M type=write size=3748617 2024-03-04T17:50:40.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.385+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=GPRFTLHTJMX8 type=del 2024-03-04T17:50:40.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.389+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~UYFKW24JBGUO.0 size=1048576 2024-03-04T17:50:40.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.393+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~UYFKW24JBGUO.1 size=563734 2024-03-04T17:50:40.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.393+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=UYFKW24JBGUO tag=0Q04QIO9IXKB type=write size=2660886 2024-03-04T17:50:40.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.401+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~WLQVD3QXKG8K.0 size=1048576 2024-03-04T17:50:40.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.401+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~WLQVD3QXKG8K.1 size=1048576 2024-03-04T17:50:40.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.405+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~WLQVD3QXKG8K.2 size=1034210 2024-03-04T17:50:40.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.413+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=WLQVD3QXKG8K tag=OJYPMKQWD1GR type=write size=4179938 2024-03-04T17:50:40.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.417+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=1JZGPOR0ZNWK type=del size=1259084 2024-03-04T17:50:40.420 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.421+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=NF3HNRKK41IU type=del 2024-03-04T17:50:40.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.425+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=R1CO5W1ZN16S type=write size=97056 2024-03-04T17:50:40.428 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.429+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~WRVAF083WVB8.0 size=1048576 2024-03-04T17:50:40.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.433+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~WRVAF083WVB8.1 size=1048576 2024-03-04T17:50:40.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.437+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~WRVAF083WVB8.2 size=1026816 2024-03-04T17:50:40.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.441+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=WRVAF083WVB8 tag=AYLG7UZMB2VN type=write size=4172544 2024-03-04T17:50:40.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.445+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=8LO3IG15GK35 type=write size=632106 2024-03-04T17:50:40.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.449+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=RP6PY6Z6S9PT type=del 2024-03-04T17:50:40.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.453+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=T6Y2JZDH98ZY type=del size=3887731 2024-03-04T17:50:40.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.457+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~G5M9OECTJ2II.0 size=1048576 2024-03-04T17:50:40.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.461+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~G5M9OECTJ2II.1 size=1048576 2024-03-04T17:50:40.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.465+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~G5M9OECTJ2II.2 size=77343 2024-03-04T17:50:40.469 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.469+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=G5M9OECTJ2II tag=WYK3961UEGMC type=write size=3223071 2024-03-04T17:50:40.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.473+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=MSY75OHRP53T type=del size=3719975 2024-03-04T17:50:40.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.477+0000 7fda08af98c0 -1 < no bi entry to suggest for operation key=09KBCA2XURTF tag=1XTQG5CACXCR type=del size=1666059: (2) No such file or directory 2024-03-04T17:50:40.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.481+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=4D31HV3NDOY7 type=del size=795206 2024-03-04T17:50:40.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.485+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~3YAVM1AV9UVB.0 size=621051 2024-03-04T17:50:40.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.489+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=3YAVM1AV9UVB tag=Y2I1EKSVJQG6 type=write size=1669627 2024-03-04T17:50:40.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.493+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=PANZ113759WS type=del size=985394 2024-03-04T17:50:40.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.497+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~CR9Q0LBXWZQ9.0 size=348987 2024-03-04T17:50:40.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.505+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=CR9Q0LBXWZQ9 tag=GVB3BYE8PUKI type=write size=1397563 2024-03-04T17:50:40.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.509+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=WK0MHOQI2GY8 type=del size=2186217 2024-03-04T17:50:40.509 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.513+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~YBEH7G3RENCQ.0 size=1048576 2024-03-04T17:50:40.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.513+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~YBEH7G3RENCQ.1 size=1048576 2024-03-04T17:50:40.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.517+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~YBEH7G3RENCQ.2 size=135793 2024-03-04T17:50:40.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.525+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=YBEH7G3RENCQ tag=XP7SZ96GXY67 type=write size=3281521 2024-03-04T17:50:40.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.529+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~WUGT40RJOSE7.0 size=1048576 2024-03-04T17:50:40.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.533+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~WUGT40RJOSE7.1 size=1048576 2024-03-04T17:50:40.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.537+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~WUGT40RJOSE7.2 size=325095 2024-03-04T17:50:40.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.537+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=WUGT40RJOSE7 tag=EOSNDV1V9AE9 type=write size=3470823 2024-03-04T17:50:40.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.541+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=UO8PXWZE1VBJ type=del 2024-03-04T17:50:40.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.541+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=USGXV55P20TV type=del size=51795 2024-03-04T17:50:40.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.545+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~VA8KK6F63ZYY.0 size=1048576 2024-03-04T17:50:40.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.549+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~VA8KK6F63ZYY.1 size=1048576 2024-03-04T17:50:40.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.553+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~VA8KK6F63ZYY.2 size=754129 2024-03-04T17:50:40.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.561+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=VA8KK6F63ZYY tag=C5175AX626QM type=write size=3899857 2024-03-04T17:50:40.563 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.565+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=JMU5LISNNOTW type=del size=1086456 2024-03-04T17:50:40.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.569+0000 7fda08af98c0 1 < canceled operation key=VJVORUG95FLC tag=AQW4N2ISJP57 type=write size=58146 2024-03-04T17:50:40.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.573+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=Z79P0HVXX6RG type=del size=1026205 2024-03-04T17:50:40.573 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.573+0000 7fda08af98c0 1 < canceled part key=_multipart_Z2JQBN84QQIB.2~K8C3EAMZ12XU.0 size=1048576 2024-03-04T17:50:40.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.577+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~K8C3EAMZ12XU.1 size=1029188 2024-03-04T17:50:40.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.585+0000 7fda08af98c0 1 < canceled multipart upload key=Z2JQBN84QQIB upload=K8C3EAMZ12XU tag=8FU0TB3SOBWT type=write size=3126340 2024-03-04T17:50:40.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.589+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=RXXN3VAQ5CL1 type=del size=1812394 2024-03-04T17:50:40.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.601+0000 7fda08af98c0 1 < suggested remove operation key=8YL7C9PI58TK tag=HLMDGA010BNX type=del 2024-03-04T17:50:40.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.605+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~E2DLW3UEVDX4.0 size=1048576 2024-03-04T17:50:40.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.613+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~E2DLW3UEVDX4.1 size=482859 2024-03-04T17:50:40.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.613+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=E2DLW3UEVDX4 tag=AF4JJB80TMTL type=write size=2580011 2024-03-04T17:50:40.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.617+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~LJVH2146EGL5.0 size=1048576 2024-03-04T17:50:40.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.621+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~LJVH2146EGL5.1 size=1048576 2024-03-04T17:50:40.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.625+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~LJVH2146EGL5.2 size=602889 2024-03-04T17:50:40.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.633+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=LJVH2146EGL5 tag=BYSLR3KGFR2M type=write size=3748617 2024-03-04T17:50:40.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.637+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=F3OELXHN42EY type=del size=1030720 2024-03-04T17:50:40.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.641+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~UYFKW24JBGUO.0 size=1048576 2024-03-04T17:50:40.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.641+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~UYFKW24JBGUO.1 size=563734 2024-03-04T17:50:40.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.645+0000 7fda08af98c0 1 < canceled multipart upload key=JA4JK99I0942 upload=UYFKW24JBGUO tag=0Q04QIO9IXKB type=write size=2660886 2024-03-04T17:50:40.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.653+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~WVVWQ109GTLR.0 size=1048576 2024-03-04T17:50:40.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.657+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~WVVWQ109GTLR.1 size=1048576 2024-03-04T17:50:40.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.661+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~WVVWQ109GTLR.2 size=1027074 2024-03-04T17:50:40.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.661+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=WVVWQ109GTLR tag=5IQ5AALT2796 type=write size=4172802 2024-03-04T17:50:40.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.669+0000 7fda08af98c0 1 < suggested update operation key=1OZKPHMPV3OQ tag=1JZGPOR0ZNWK type=del size=1259084 2024-03-04T17:50:40.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.673+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=92PRD65S1CSU type=del size=1979020 2024-03-04T17:50:40.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.677+0000 7fda08af98c0 1 < completed write operation key=RXVSQVVNXVF0 tag=R1CO5W1ZN16S type=write size=97056 2024-03-04T17:50:40.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.681+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=83IYTSYKQ7BT type=del size=1016750 2024-03-04T17:50:40.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.689+0000 7fda08af98c0 1 < suggested remove operation key=0IGSMFORCSL9 tag=8LO3IG15GK35 type=write 2024-03-04T17:50:40.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.697+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~4A73VR7DO1G9.0 size=1048576 2024-03-04T17:50:40.697 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.697+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~4A73VR7DO1G9.1 size=1048576 2024-03-04T17:50:40.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.701+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~4A73VR7DO1G9.2 size=1010500 2024-03-04T17:50:40.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.701+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=4A73VR7DO1G9 tag=KJORDT9TFMF8 type=write size=4156228 2024-03-04T17:50:40.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.705+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=T6Y2JZDH98ZY type=del 2024-03-04T17:50:40.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.709+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=CX3RWY21ECWV type=write size=341031 2024-03-04T17:50:40.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.713+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=MSY75OHRP53T type=del 2024-03-04T17:50:40.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.717+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=80OSPYZ4NEDN type=del size=2452807 2024-03-04T17:50:40.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.721+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=4D31HV3NDOY7 type=del 2024-03-04T17:50:40.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.725+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=LQLXC06308NW type=del size=903462 2024-03-04T17:50:40.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.733+0000 7fda08af98c0 1 < suggested update operation key=2PA8JDZUOLO5 tag=PANZ113759WS type=del size=985394 2024-03-04T17:50:40.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.737+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=4WM1SHC3KA6Z type=del size=1586927 2024-03-04T17:50:40.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.741+0000 7fda08af98c0 -1 < failed to complete operation key=0IGSMFORCSL9 tag=WK0MHOQI2GY8 type=del size=2186217: (22) Invalid argument 2024-03-04T17:50:40.743 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.745+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=L2ZTH42FWKM2 type=write size=286880 2024-03-04T17:50:40.747 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.749+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~WUGT40RJOSE7.0 size=1048576 2024-03-04T17:50:40.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.753+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~WUGT40RJOSE7.1 size=1048576 2024-03-04T17:50:40.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.757+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~WUGT40RJOSE7.2 size=325095 2024-03-04T17:50:40.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.761+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=WUGT40RJOSE7 tag=EOSNDV1V9AE9 type=write size=3470823 2024-03-04T17:50:40.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.769+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~FYABIDU134WR.0 size=1048576 2024-03-04T17:50:40.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.773+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~FYABIDU134WR.1 size=1048576 2024-03-04T17:50:40.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.773+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~FYABIDU134WR.2 size=204154 2024-03-04T17:50:40.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.773+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=FYABIDU134WR tag=OSVT00PFZAHW type=write size=3349882 2024-03-04T17:50:40.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.777+0000 7fda08af98c0 -1 < failed to complete operation key=2PA8JDZUOLO5 tag=USGXV55P20TV type=del size=51795: (22) Invalid argument 2024-03-04T17:50:40.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.785+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~0NIAGQWS19HS.0 size=1048576 2024-03-04T17:50:40.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.785+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~0NIAGQWS19HS.1 size=1048576 2024-03-04T17:50:40.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.789+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~0NIAGQWS19HS.2 size=364393 2024-03-04T17:50:40.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.789+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=0NIAGQWS19HS tag=257QDWTL8IFN type=write size=3510121 2024-03-04T17:50:40.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.793+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=JMU5LISNNOTW type=del 2024-03-04T17:50:40.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.797+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=HOF30TP53BZB type=del size=1093189 2024-03-04T17:50:40.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.801+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=Z79P0HVXX6RG type=del 2024-03-04T17:50:40.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.809+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~WBO7E8EL1Z51.0 size=1048576 2024-03-04T17:50:40.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.813+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~WBO7E8EL1Z51.1 size=1048576 2024-03-04T17:50:40.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.817+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~WBO7E8EL1Z51.2 size=1031825 2024-03-04T17:50:40.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.817+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=WBO7E8EL1Z51 tag=PR7GZUKVRC6P type=write size=4177553 2024-03-04T17:50:40.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.825+0000 7fda08af98c0 1 < suggested update operation key=3UWSL7NJLACY tag=RXXN3VAQ5CL1 type=del size=1812394 2024-03-04T17:50:40.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.829+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=Z1BHY6VIAHWY type=del size=2807395 2024-03-04T17:50:40.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.833+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~E2DLW3UEVDX4.0 size=1048576 2024-03-04T17:50:40.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.833+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~E2DLW3UEVDX4.1 size=482859 2024-03-04T17:50:40.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.841+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=E2DLW3UEVDX4 tag=AF4JJB80TMTL type=write size=2580011 2024-03-04T17:50:40.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.849+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~7H9EETCRG2NH.0 size=1048576 2024-03-04T17:50:40.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.849+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~7H9EETCRG2NH.1 size=717357 2024-03-04T17:50:40.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.849+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=7H9EETCRG2NH tag=VYVG9HBVDVNA type=write size=2814509 2024-03-04T17:50:40.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.853+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=F3OELXHN42EY type=del 2024-03-04T17:50:40.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.857+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=EAD9JGQVY48N type=del size=1450241 2024-03-04T17:50:40.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.861+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~WVVWQ109GTLR.0 size=1048576 2024-03-04T17:50:40.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.865+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~WVVWQ109GTLR.1 size=1048576 2024-03-04T17:50:40.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.869+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~WVVWQ109GTLR.2 size=1027074 2024-03-04T17:50:40.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.877+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=WVVWQ109GTLR tag=5IQ5AALT2796 type=write size=4172802 2024-03-04T17:50:40.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.881+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~HNLAODFMRYKX.0 size=1048576 2024-03-04T17:50:40.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.885+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~HNLAODFMRYKX.1 size=1048576 2024-03-04T17:50:40.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.889+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~HNLAODFMRYKX.2 size=961936 2024-03-04T17:50:40.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.889+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=HNLAODFMRYKX tag=EQ429N4G7Y3B type=write size=4107664 2024-03-04T17:50:40.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.893+0000 7fda08af98c0 -1 < failed to complete operation key=3UWSL7NJLACY tag=92PRD65S1CSU type=del size=1979020: (22) Invalid argument 2024-03-04T17:50:40.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.897+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=AX0N0GJJQP5W type=del size=69963 2024-03-04T17:50:40.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.901+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=83IYTSYKQ7BT type=del 2024-03-04T17:50:40.906 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.909+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~S66WJA0S3C6T.0 size=786893 2024-03-04T17:50:40.906 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.909+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=S66WJA0S3C6T tag=RR8HD77F9V52 type=write size=1835469 2024-03-04T17:50:40.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.913+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~4A73VR7DO1G9.0 size=1048576 2024-03-04T17:50:40.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.913+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~4A73VR7DO1G9.1 size=1048576 2024-03-04T17:50:40.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.917+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~4A73VR7DO1G9.2 size=1010500 2024-03-04T17:50:40.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.925+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=4A73VR7DO1G9 tag=KJORDT9TFMF8 type=write size=4156228 2024-03-04T17:50:40.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.933+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~YL3QKJYP8VXJ.0 size=1048576 2024-03-04T17:50:40.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.933+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~YL3QKJYP8VXJ.1 size=324084 2024-03-04T17:50:40.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.933+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=YL3QKJYP8VXJ tag=JZ52W3GVBHQ9 type=write size=2421236 2024-03-04T17:50:40.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.941+0000 7fda08af98c0 1 < suggested update operation key=Z2JQBN84QQIB tag=CX3RWY21ECWV type=write size=341031 2024-03-04T17:50:40.944 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.945+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=S9HAGRSDDJR3 type=write size=56926 2024-03-04T17:50:40.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.949+0000 7fda08af98c0 -1 < failed to complete operation key=Z2JQBN84QQIB tag=80OSPYZ4NEDN type=del size=2452807: (22) Invalid argument 2024-03-04T17:50:40.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.953+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=AVCFXZD1B0KH type=del size=2164236 2024-03-04T17:50:40.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.957+0000 7fda08af98c0 -1 < failed to complete operation key=3UWSL7NJLACY tag=LQLXC06308NW type=del size=903462: (22) Invalid argument 2024-03-04T17:50:40.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.965+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~W5CHIHY4N8ZO.0 size=887503 2024-03-04T17:50:40.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.965+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=W5CHIHY4N8ZO tag=KU8E85K267E5 type=write size=1936079 2024-03-04T17:50:40.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.969+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=4WM1SHC3KA6Z type=del 2024-03-04T17:50:40.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.977+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~RLV6VJTRV2PO.0 size=1048576 2024-03-04T17:50:40.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.981+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~RLV6VJTRV2PO.1 size=176974 2024-03-04T17:50:40.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.981+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=RLV6VJTRV2PO tag=FNAJLVGOYL2H type=write size=2274126 2024-03-04T17:50:40.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.985+0000 7fda08af98c0 -1 < failed to complete operation key=3UWSL7NJLACY tag=L2ZTH42FWKM2 type=write size=286880: (22) Invalid argument 2024-03-04T17:50:40.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.989+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=LA2DH7AFFJY4 type=del size=3278163 2024-03-04T17:50:40.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.993+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~FYABIDU134WR.0 size=1048576 2024-03-04T17:50:40.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:40.997+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~FYABIDU134WR.1 size=1048576 2024-03-04T17:50:40.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.001+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~FYABIDU134WR.2 size=204154 2024-03-04T17:50:41.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.005+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=FYABIDU134WR tag=OSVT00PFZAHW type=write size=3349882 2024-03-04T17:50:41.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.009+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=8N44DY5V5AQU type=del size=147807 2024-03-04T17:50:41.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.013+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~0NIAGQWS19HS.0 size=1048576 2024-03-04T17:50:41.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.017+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~0NIAGQWS19HS.1 size=1048576 2024-03-04T17:50:41.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.021+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~0NIAGQWS19HS.2 size=364393 2024-03-04T17:50:41.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.025+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=0NIAGQWS19HS tag=257QDWTL8IFN type=write size=3510121 2024-03-04T17:50:41.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.033+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~CD6WUGQ021UK.0 size=1048576 2024-03-04T17:50:41.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.037+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~CD6WUGQ021UK.1 size=370371 2024-03-04T17:50:41.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.037+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=CD6WUGQ021UK tag=PNA9ZWITSMD0 type=write size=2467523 2024-03-04T17:50:41.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.045+0000 7fda08af98c0 1 < suggested remove operation key=VJVORUG95FLC tag=HOF30TP53BZB type=del 2024-03-04T17:50:41.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.049+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=0QRF0UKLKTBP type=del size=3802387 2024-03-04T17:50:41.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.053+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~WBO7E8EL1Z51.0 size=1048576 2024-03-04T17:50:41.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.057+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~WBO7E8EL1Z51.1 size=1048576 2024-03-04T17:50:41.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.061+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~WBO7E8EL1Z51.2 size=1031825 2024-03-04T17:50:41.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.065+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=WBO7E8EL1Z51 tag=PR7GZUKVRC6P type=write size=4177553 2024-03-04T17:50:41.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.069+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=8A1JRE01LN19 type=del size=421201 2024-03-04T17:50:41.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.073+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=Z1BHY6VIAHWY type=del 2024-03-04T17:50:41.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.081+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~54W4LWZEPP6W.0 size=228421 2024-03-04T17:50:41.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.081+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=54W4LWZEPP6W tag=PWW1Q1P8BTFD type=write size=1276997 2024-03-04T17:50:41.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.085+0000 7fda08af98c0 1 < canceled part key=_multipart_ITIVF3UAT9NK.2~7H9EETCRG2NH.0 size=1048576 2024-03-04T17:50:41.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.085+0000 7fda08af98c0 1 < canceled part key=_multipart_ITIVF3UAT9NK.2~7H9EETCRG2NH.1 size=717357 2024-03-04T17:50:41.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.089+0000 7fda08af98c0 1 < canceled multipart upload key=ITIVF3UAT9NK upload=7H9EETCRG2NH tag=VYVG9HBVDVNA type=write size=2814509 2024-03-04T17:50:41.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.093+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=J1JIG30DSG0V type=del size=2897935 2024-03-04T17:50:41.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.097+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=EAD9JGQVY48N type=del 2024-03-04T17:50:41.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.101+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=BWOL5E1P2B6O type=del size=872233 2024-03-04T17:50:41.103 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.105+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~HNLAODFMRYKX.0 size=1048576 2024-03-04T17:50:41.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.109+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~HNLAODFMRYKX.1 size=1048576 2024-03-04T17:50:41.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.113+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~HNLAODFMRYKX.2 size=961936 2024-03-04T17:50:41.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.117+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=HNLAODFMRYKX tag=EQ429N4G7Y3B type=write size=4107664 2024-03-04T17:50:41.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.121+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=PCRUG743YZPL type=del size=550139 2024-03-04T17:50:41.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.129+0000 7fda08af98c0 1 < suggested update operation key=HCA7QLGM2O4O tag=AX0N0GJJQP5W type=del size=69963 2024-03-04T17:50:41.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.133+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=Q36O9NCTDYSP type=del size=120896 2024-03-04T17:50:41.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.137+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~S66WJA0S3C6T.0 size=786893 2024-03-04T17:50:41.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.141+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=S66WJA0S3C6T tag=RR8HD77F9V52 type=write size=1835469 2024-03-04T17:50:41.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.149+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~YMR4HCL67MQB.0 size=1048576 2024-03-04T17:50:41.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.153+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~YMR4HCL67MQB.1 size=1048576 2024-03-04T17:50:41.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.153+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~YMR4HCL67MQB.2 size=582101 2024-03-04T17:50:41.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.153+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=YMR4HCL67MQB tag=T0GGW294ZKJX type=write size=3727829 2024-03-04T17:50:41.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.157+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~YL3QKJYP8VXJ.0 size=1048576 2024-03-04T17:50:41.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.161+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~YL3QKJYP8VXJ.1 size=324084 2024-03-04T17:50:41.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.169+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=YL3QKJYP8VXJ tag=JZ52W3GVBHQ9 type=write size=2421236 2024-03-04T17:50:41.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.169+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=AA7IR88EYLK0 type=write size=766955 2024-03-04T17:50:41.176 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.177+0000 7fda08af98c0 1 < suggested remove operation key=JA4JK99I0942 tag=S9HAGRSDDJR3 type=write 2024-03-04T17:50:41.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.181+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=57SW07SH09VF type=del size=3493083 2024-03-04T17:50:41.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.185+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=AVCFXZD1B0KH type=del 2024-03-04T17:50:41.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.193+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~U0ZTDP7TURV4.0 size=1048576 2024-03-04T17:50:41.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.197+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~U0ZTDP7TURV4.1 size=683709 2024-03-04T17:50:41.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.197+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=U0ZTDP7TURV4 tag=P8EAPSE3KVMU type=write size=2780861 2024-03-04T17:50:41.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.201+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~W5CHIHY4N8ZO.0 size=887503 2024-03-04T17:50:41.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.205+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=W5CHIHY4N8ZO tag=KU8E85K267E5 type=write size=1936079 2024-03-04T17:50:41.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.209+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=69OGM95SOKJ0 type=del size=2425279 2024-03-04T17:50:41.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.213+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~RLV6VJTRV2PO.0 size=1048576 2024-03-04T17:50:41.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.213+0000 7fda08af98c0 1 < canceled part key=_multipart_AO3QS6CM7I5D.2~RLV6VJTRV2PO.1 size=176974 2024-03-04T17:50:41.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.221+0000 7fda08af98c0 1 < canceled multipart upload key=AO3QS6CM7I5D upload=RLV6VJTRV2PO tag=FNAJLVGOYL2H type=write size=2274126 2024-03-04T17:50:41.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.225+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=8ZOB5VHTXIJI type=del size=378718 2024-03-04T17:50:41.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.229+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=LA2DH7AFFJY4 type=del 2024-03-04T17:50:41.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.233+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=VYY5UPK0IL62 type=del size=713470 2024-03-04T17:50:41.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.237+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=8N44DY5V5AQU type=del 2024-03-04T17:50:41.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.241+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=KKQ7WW6KAXVA type=del size=3353658 2024-03-04T17:50:41.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.245+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~CD6WUGQ021UK.0 size=1048576 2024-03-04T17:50:41.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.245+0000 7fda08af98c0 1 < canceled part key=_multipart_HCA7QLGM2O4O.2~CD6WUGQ021UK.1 size=370371 2024-03-04T17:50:41.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.253+0000 7fda08af98c0 1 < canceled multipart upload key=HCA7QLGM2O4O upload=CD6WUGQ021UK tag=PNA9ZWITSMD0 type=write size=2467523 2024-03-04T17:50:41.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.257+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=8MZAKOHNLYYX type=del size=3832961 2024-03-04T17:50:41.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.261+0000 7fda08af98c0 1 < canceled operation key=GDGTJZ21TG32 tag=0QRF0UKLKTBP type=del size=3802387 2024-03-04T17:50:41.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.269+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~MPZCGCS37FZT.0 size=1048576 2024-03-04T17:50:41.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.273+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~MPZCGCS37FZT.1 size=973601 2024-03-04T17:50:41.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.273+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=MPZCGCS37FZT tag=QMTJIGF8613Q type=write size=3070753 2024-03-04T17:50:41.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.277+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=8A1JRE01LN19 type=del 2024-03-04T17:50:41.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.281+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=IA4Z2CJ7DT2Z type=del size=1665008 2024-03-04T17:50:41.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.285+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~54W4LWZEPP6W.0 size=228421 2024-03-04T17:50:41.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.293+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=54W4LWZEPP6W tag=PWW1Q1P8BTFD type=write size=1276997 2024-03-04T17:50:41.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.301+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~IXUDUNSDQ4UM.0 size=966097 2024-03-04T17:50:41.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.301+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=IXUDUNSDQ4UM tag=A8Y6DIUUD2CW type=write size=2014673 2024-03-04T17:50:41.308 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.309+0000 7fda08af98c0 1 < suggested remove operation key=HCKJZUMAK0Z0 tag=J1JIG30DSG0V type=del 2024-03-04T17:50:41.312 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.313+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=L5JEBRC3DTLM type=del size=1844464 2024-03-04T17:50:41.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.317+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=BWOL5E1P2B6O type=del 2024-03-04T17:50:41.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.325+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~VYA098UUV63C.0 size=459994 2024-03-04T17:50:41.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.325+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=VYA098UUV63C tag=T7BBN91QTU8H type=write size=1508570 2024-03-04T17:50:41.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.333+0000 7fda08af98c0 1 < suggested update operation key=3UWSL7NJLACY tag=PCRUG743YZPL type=del size=550139 2024-03-04T17:50:41.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.341+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=WJ7Y3VXM21WR type=del size=2316786 2024-03-04T17:50:41.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.349+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=Q36O9NCTDYSP type=del 2024-03-04T17:50:41.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.353+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=JVD7VQ50SSTU type=del size=2143772 2024-03-04T17:50:41.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.357+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~YMR4HCL67MQB.0 size=1048576 2024-03-04T17:50:41.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.361+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~YMR4HCL67MQB.1 size=1048576 2024-03-04T17:50:41.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.365+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~YMR4HCL67MQB.2 size=582101 2024-03-04T17:50:41.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.373+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=YMR4HCL67MQB tag=T0GGW294ZKJX type=write size=3727829 2024-03-04T17:50:41.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.377+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=6ZF8LI0DMIOK type=del size=17882 2024-03-04T17:50:41.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.381+0000 7fda08af98c0 1 < completed write operation key=GDGTJZ21TG32 tag=AA7IR88EYLK0 type=write size=766955 2024-03-04T17:50:41.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.385+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=UOY8GUP862OI type=write size=859415 2024-03-04T17:50:41.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.389+0000 7fda08af98c0 1 < canceled operation key=RXVSQVVNXVF0 tag=57SW07SH09VF type=del size=3493083 2024-03-04T17:50:41.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.397+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~K0RR0ULE8C9O.0 size=270131 2024-03-04T17:50:41.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.397+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=K0RR0ULE8C9O tag=LTXGBNAZ7ZD9 type=write size=1318707 2024-03-04T17:50:41.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.401+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~U0ZTDP7TURV4.0 size=1048576 2024-03-04T17:50:41.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.405+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~U0ZTDP7TURV4.1 size=683709 2024-03-04T17:50:41.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.413+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=U0ZTDP7TURV4 tag=P8EAPSE3KVMU type=write size=2780861 2024-03-04T17:50:41.414 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.417+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=YMZC11NGDDJ2 type=del size=1501162 2024-03-04T17:50:41.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.421+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=69OGM95SOKJ0 type=del 2024-03-04T17:50:41.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.425+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=7R37NWP0BZ07 type=del size=2032157 2024-03-04T17:50:41.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.429+0000 7fda08af98c0 1 < suggested update operation key=TOFWDFH0HL3Q tag=8ZOB5VHTXIJI type=del size=378718 2024-03-04T17:50:41.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.437+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=CL6UZGRZ6U7D type=del size=3526540 2024-03-04T17:50:41.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.441+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=VYY5UPK0IL62 type=del 2024-03-04T17:50:41.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.445+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=UXCVLG8QJH84 type=del size=1779854 2024-03-04T17:50:41.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.449+0000 7fda08af98c0 1 < suggested update operation key=0IGSMFORCSL9 tag=KKQ7WW6KAXVA type=del size=3353658 2024-03-04T17:50:41.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.453+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=9EGCMFS227A8 type=del size=3678156 2024-03-04T17:50:41.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.457+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=8MZAKOHNLYYX type=del 2024-03-04T17:50:41.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.465+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~NTB0W17QAYTF.0 size=809974 2024-03-04T17:50:41.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.465+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=NTB0W17QAYTF tag=N3Q58FL71ZXV type=write size=1858550 2024-03-04T17:50:41.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.469+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~MPZCGCS37FZT.0 size=1048576 2024-03-04T17:50:41.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.469+0000 7fda08af98c0 1 < canceled part key=_multipart_0IGSMFORCSL9.2~MPZCGCS37FZT.1 size=973601 2024-03-04T17:50:41.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.473+0000 7fda08af98c0 1 < canceled multipart upload key=0IGSMFORCSL9 upload=MPZCGCS37FZT tag=QMTJIGF8613Q type=write size=3070753 2024-03-04T17:50:41.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.481+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~IXDM4QTHKACQ.0 size=1048576 2024-03-04T17:50:41.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.485+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~IXDM4QTHKACQ.1 size=198049 2024-03-04T17:50:41.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.485+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=IXDM4QTHKACQ tag=33H5WWKZGYWU type=write size=2295201 2024-03-04T17:50:41.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.493+0000 7fda08af98c0 1 < suggested update operation key=750NSVFBS03A tag=IA4Z2CJ7DT2Z type=del size=1665008 2024-03-04T17:50:41.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.501+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~UOW7OB4X8XDB.0 size=181209 2024-03-04T17:50:41.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.501+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=UOW7OB4X8XDB tag=JSAJQL6BAL7K type=write size=1229785 2024-03-04T17:50:41.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.505+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~IXUDUNSDQ4UM.0 size=966097 2024-03-04T17:50:41.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.509+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=IXUDUNSDQ4UM tag=A8Y6DIUUD2CW type=write size=2014673 2024-03-04T17:50:41.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.513+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=PRLKWPZQAXF7 type=write size=157114 2024-03-04T17:50:41.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.517+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=L5JEBRC3DTLM type=del 2024-03-04T17:50:41.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.521+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=S92K3XBUQ0RA type=del size=3823877 2024-03-04T17:50:41.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.525+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~VYA098UUV63C.0 size=459994 2024-03-04T17:50:41.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.533+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=VYA098UUV63C tag=T7BBN91QTU8H type=write size=1508570 2024-03-04T17:50:41.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.537+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~UKHB7ODVZ0QC.0 size=752419 2024-03-04T17:50:41.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.537+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=UKHB7ODVZ0QC tag=25AEEQPKHSAJ type=write size=1800995 2024-03-04T17:50:41.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.541+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=WJ7Y3VXM21WR type=del 2024-03-04T17:50:41.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.549+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~ZDUZVAUK2NKG.0 size=1048576 2024-03-04T17:50:41.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.553+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~ZDUZVAUK2NKG.1 size=531150 2024-03-04T17:50:41.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.553+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=ZDUZVAUK2NKG tag=TTV37Y60E149 type=write size=2628302 2024-03-04T17:50:41.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.557+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=JVD7VQ50SSTU type=del 2024-03-04T17:50:41.559 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.561+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=UBX54SUJE329 type=del size=1837151 2024-03-04T17:50:41.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.569+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=6ZF8LI0DMIOK type=del 2024-03-04T17:50:41.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.577+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~N6AQ6U2F8WZK.0 size=956447 2024-03-04T17:50:41.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.577+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=N6AQ6U2F8WZK tag=JWUGPM1G1NR2 type=write size=2005023 2024-03-04T17:50:41.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.581+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=UOY8GUP862OI type=write size=859415 2024-03-04T17:50:41.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.585+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=9FQO3C3XP2BZ type=del size=4007121 2024-03-04T17:50:41.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.589+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~K0RR0ULE8C9O.0 size=270131 2024-03-04T17:50:41.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.593+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=K0RR0ULE8C9O tag=LTXGBNAZ7ZD9 type=write size=1318707 2024-03-04T17:50:41.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.597+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=GX14B1M3NRLE type=write size=565908 2024-03-04T17:50:41.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.605+0000 7fda08af98c0 1 < suggested remove operation key=VJVORUG95FLC tag=YMZC11NGDDJ2 type=del 2024-03-04T17:50:41.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.609+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=0OZIPGVVARA2 type=del size=1054513 2024-03-04T17:50:41.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.617+0000 7fda08af98c0 1 < suggested remove operation key=Z3J7W9V8HQKH tag=7R37NWP0BZ07 type=del 2024-03-04T17:50:41.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.625+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~J9RSFYLCIT4P.0 size=1048576 2024-03-04T17:50:41.626 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.629+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~J9RSFYLCIT4P.1 size=1027183 2024-03-04T17:50:41.626 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.629+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=J9RSFYLCIT4P tag=W87USBZ4W3D1 type=write size=3124335 2024-03-04T17:50:41.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.633+0000 7fda08af98c0 1 < canceled operation key=RXVSQVVNXVF0 tag=CL6UZGRZ6U7D type=del size=3526540 2024-03-04T17:50:41.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.637+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=YAGO7OEPGNXT type=del size=388144 2024-03-04T17:50:41.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.641+0000 7fda08af98c0 1 < suggested remove operation key=C1GK00FM2ZYA tag=UXCVLG8QJH84 type=del 2024-03-04T17:50:41.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.649+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~0OUCSRUHNII6.0 size=1048576 2024-03-04T17:50:41.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.653+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~0OUCSRUHNII6.1 size=1048576 2024-03-04T17:50:41.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.657+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~0OUCSRUHNII6.2 size=790112 2024-03-04T17:50:41.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.657+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=0OUCSRUHNII6 tag=18EQIK2G2M4S type=write size=3935840 2024-03-04T17:50:41.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.661+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=9EGCMFS227A8 type=del 2024-03-04T17:50:41.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.665+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=TRAGJ6OVRV51 type=del size=3842844 2024-03-04T17:50:41.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.669+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~NTB0W17QAYTF.0 size=809974 2024-03-04T17:50:41.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.677+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=NTB0W17QAYTF tag=N3Q58FL71ZXV type=write size=1858550 2024-03-04T17:50:41.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.681+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=I0XZG8AMT2CT type=del size=3964911 2024-03-04T17:50:41.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.685+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~IXDM4QTHKACQ.0 size=1048576 2024-03-04T17:50:41.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.685+0000 7fda08af98c0 1 < canceled part key=_multipart_AO3QS6CM7I5D.2~IXDM4QTHKACQ.1 size=198049 2024-03-04T17:50:41.689 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.693+0000 7fda08af98c0 1 < canceled multipart upload key=AO3QS6CM7I5D upload=IXDM4QTHKACQ tag=33H5WWKZGYWU type=write size=2295201 2024-03-04T17:50:41.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.693+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=GJZK4USNODSB type=del size=768605 2024-03-04T17:50:41.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.701+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~UOW7OB4X8XDB.0 size=181209 2024-03-04T17:50:41.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.705+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=UOW7OB4X8XDB tag=JSAJQL6BAL7K type=write size=1229785 2024-03-04T17:50:41.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.713+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~NX5W7RVFUWME.0 size=1048576 2024-03-04T17:50:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.717+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~NX5W7RVFUWME.1 size=1048576 2024-03-04T17:50:41.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.721+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~NX5W7RVFUWME.2 size=734371 2024-03-04T17:50:41.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.721+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=NX5W7RVFUWME tag=2FQ4B3ZTSM4F type=write size=3880099 2024-03-04T17:50:41.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.725+0000 7fda08af98c0 1 < completed write operation key=BYQFZSJ3WH07 tag=PRLKWPZQAXF7 type=write size=157114 2024-03-04T17:50:41.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.729+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=L0ZACMELDH38 type=del size=328023 2024-03-04T17:50:41.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.733+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=S92K3XBUQ0RA type=del 2024-03-04T17:50:41.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.741+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~7V9OQ0TEHWX7.0 size=1048576 2024-03-04T17:50:41.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.741+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~7V9OQ0TEHWX7.1 size=109607 2024-03-04T17:50:41.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.741+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=7V9OQ0TEHWX7 tag=F51O0E2RAPZF type=write size=2206759 2024-03-04T17:50:41.744 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.745+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~UKHB7ODVZ0QC.0 size=752419 2024-03-04T17:50:41.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.753+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=UKHB7ODVZ0QC tag=25AEEQPKHSAJ type=write size=1800995 2024-03-04T17:50:41.754 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.757+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=30NYCUASF1E5 type=write size=780821 2024-03-04T17:50:41.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.761+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~ZDUZVAUK2NKG.0 size=1048576 2024-03-04T17:50:41.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.761+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~ZDUZVAUK2NKG.1 size=531150 2024-03-04T17:50:41.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.769+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=ZDUZVAUK2NKG tag=TTV37Y60E149 type=write size=2628302 2024-03-04T17:50:41.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.773+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=SXBQ3R59FCVE type=del size=1893350 2024-03-04T17:50:41.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.777+0000 7fda08af98c0 1 < canceled operation key=RXVSQVVNXVF0 tag=UBX54SUJE329 type=del size=1837151 2024-03-04T17:50:41.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.781+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=G5O6XP7VAWTR type=del size=1088154 2024-03-04T17:50:41.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.785+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~N6AQ6U2F8WZK.0 size=956447 2024-03-04T17:50:41.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.789+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=N6AQ6U2F8WZK tag=JWUGPM1G1NR2 type=write size=2005023 2024-03-04T17:50:41.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.793+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=KPWMLBWMZYB6 type=del size=1205508 2024-03-04T17:50:41.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.801+0000 7fda08af98c0 1 < suggested update operation key=XG5I3SDRE5JM tag=9FQO3C3XP2BZ type=del size=4007121 2024-03-04T17:50:41.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.805+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=UIYPJZB62D2N type=del size=440377 2024-03-04T17:50:41.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.813+0000 7fda08af98c0 1 < suggested update operation key=XG2XYBCMUNY5 tag=GX14B1M3NRLE type=write size=565908 2024-03-04T17:50:41.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.817+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=4PP7N3YVFIEN type=write size=216953 2024-03-04T17:50:41.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.821+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=0OZIPGVVARA2 type=del 2024-03-04T17:50:41.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.825+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=70N4VOI5MBVZ type=del size=1910941 2024-03-04T17:50:41.827 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.829+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~J9RSFYLCIT4P.0 size=1048576 2024-03-04T17:50:41.831 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.833+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~J9RSFYLCIT4P.1 size=1027183 2024-03-04T17:50:41.838 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.841+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=J9RSFYLCIT4P tag=W87USBZ4W3D1 type=write size=3124335 2024-03-04T17:50:41.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.849+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~CI9HGZJLJU3O.0 size=141948 2024-03-04T17:50:41.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.849+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=CI9HGZJLJU3O tag=ZGVO3UOVQSTL type=write size=1190524 2024-03-04T17:50:41.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.857+0000 7fda08af98c0 1 < suggested remove operation key=G0BHMSCO0XLF tag=YAGO7OEPGNXT type=del 2024-03-04T17:50:41.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.861+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=IKNWVJ336ZVD type=del size=471288 2024-03-04T17:50:41.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.865+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~0OUCSRUHNII6.0 size=1048576 2024-03-04T17:50:41.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.869+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~0OUCSRUHNII6.1 size=1048576 2024-03-04T17:50:41.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.869+0000 7fda08af98c0 1 < canceled part key=_multipart_FNWA2N9U14HO.2~0OUCSRUHNII6.2 size=790112 2024-03-04T17:50:41.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.873+0000 7fda08af98c0 1 < canceled multipart upload key=FNWA2N9U14HO upload=0OUCSRUHNII6 tag=18EQIK2G2M4S type=write size=3935840 2024-03-04T17:50:41.875 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.877+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=DTNQ4IBELLAW type=write size=195809 2024-03-04T17:50:41.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.881+0000 7fda08af98c0 1 < canceled operation key=JA4JK99I0942 tag=TRAGJ6OVRV51 type=del size=3842844 2024-03-04T17:50:41.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.885+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=FP3W4VBXQ8ZJ type=write size=242875 2024-03-04T17:50:41.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.889+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=I0XZG8AMT2CT type=del 2024-03-04T17:50:41.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.893+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=AJN6II6R73C8 type=del size=3497323 2024-03-04T17:50:41.894 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.897+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=GJZK4USNODSB type=del 2024-03-04T17:50:41.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.901+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~0ZTLB0TVA1OT.0 size=1048576 2024-03-04T17:50:41.904 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.905+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~0ZTLB0TVA1OT.1 size=1048576 2024-03-04T17:50:41.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.909+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~0ZTLB0TVA1OT.2 size=269725 2024-03-04T17:50:41.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.909+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=0ZTLB0TVA1OT tag=Q286I806HSC8 type=write size=3415453 2024-03-04T17:50:41.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.913+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~NX5W7RVFUWME.0 size=1048576 2024-03-04T17:50:41.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.917+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~NX5W7RVFUWME.1 size=1048576 2024-03-04T17:50:41.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.917+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~NX5W7RVFUWME.2 size=734371 2024-03-04T17:50:41.923 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.925+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=NX5W7RVFUWME tag=2FQ4B3ZTSM4F type=write size=3880099 2024-03-04T17:50:41.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.933+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~C7JQI34UB0YL.0 size=1048576 2024-03-04T17:50:41.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.937+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~C7JQI34UB0YL.1 size=1048576 2024-03-04T17:50:41.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.937+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~C7JQI34UB0YL.2 size=638071 2024-03-04T17:50:41.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.937+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=C7JQI34UB0YL tag=EC4CKKQ4JB1G type=write size=3783799 2024-03-04T17:50:41.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.941+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=L0ZACMELDH38 type=del 2024-03-04T17:50:41.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.945+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=XAQ2S1K4NKY9 type=del size=3461738 2024-03-04T17:50:41.949 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.949+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~7V9OQ0TEHWX7.0 size=1048576 2024-03-04T17:50:41.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.953+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~7V9OQ0TEHWX7.1 size=109607 2024-03-04T17:50:41.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.961+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=7V9OQ0TEHWX7 tag=F51O0E2RAPZF type=write size=2206759 2024-03-04T17:50:41.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.965+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=B3ZWKP5A66MD type=del size=1951659 2024-03-04T17:50:41.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.973+0000 7fda08af98c0 1 < suggested remove operation key=AO3QS6CM7I5D tag=30NYCUASF1E5 type=write 2024-03-04T17:50:41.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.977+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=SP0ROP14CDB2 type=del size=706657 2024-03-04T17:50:41.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.981+0000 7fda08af98c0 1 < suggested update operation key=750NSVFBS03A tag=SXBQ3R59FCVE type=del size=1893350 2024-03-04T17:50:41.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.985+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=27FLW5443WK3 type=del size=2511208 2024-03-04T17:50:41.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.989+0000 7fda08af98c0 -1 < failed to complete operation key=750NSVFBS03A tag=G5O6XP7VAWTR type=del size=1088154: (22) Invalid argument 2024-03-04T17:50:41.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:41.997+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~PH2IRUDYYY5X.0 size=1048576 2024-03-04T17:50:42.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.053+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~PH2IRUDYYY5X.1 size=1048576 2024-03-04T17:50:42.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.057+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~PH2IRUDYYY5X.2 size=942135 2024-03-04T17:50:42.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.057+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=PH2IRUDYYY5X tag=U13LHO9I1KY9 type=write size=4087863 2024-03-04T17:50:42.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.061+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=KPWMLBWMZYB6 type=del 2024-03-04T17:50:42.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.065+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=R2AE1ABKRW64 type=write size=614873 2024-03-04T17:50:42.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.069+0000 7fda08af98c0 1 < canceled operation key=L0FVKWD7EQRW tag=UIYPJZB62D2N type=del size=440377 2024-03-04T17:50:42.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.077+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~01BX8J69XWGU.0 size=1048576 2024-03-04T17:50:42.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.081+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~01BX8J69XWGU.1 size=1048576 2024-03-04T17:50:42.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.085+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~01BX8J69XWGU.2 size=796262 2024-03-04T17:50:42.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.085+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=01BX8J69XWGU tag=NTEXQAZRBACR type=write size=3941990 2024-03-04T17:50:42.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.089+0000 7fda08af98c0 -1 < failed to complete operation key=750NSVFBS03A tag=4PP7N3YVFIEN type=write size=216953: (22) Invalid argument 2024-03-04T17:50:42.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.093+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=AXHV1F03PFP0 type=write size=96160 2024-03-04T17:50:42.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.097+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=70N4VOI5MBVZ type=del 2024-03-04T17:50:42.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.101+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=XKS4BMMGZYKN type=del size=1008633 2024-03-04T17:50:42.103 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.105+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~CI9HGZJLJU3O.0 size=141948 2024-03-04T17:50:42.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.113+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=CI9HGZJLJU3O tag=ZGVO3UOVQSTL type=write size=1190524 2024-03-04T17:50:42.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.117+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=8HV1TBR4AK2P type=del size=3263040 2024-03-04T17:50:42.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.121+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=IKNWVJ336ZVD type=del 2024-03-04T17:50:42.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.125+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=DYKTK2PAPTYC type=del size=3538416 2024-03-04T17:50:42.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.129+0000 7fda08af98c0 1 < canceled operation key=L0FVKWD7EQRW tag=DTNQ4IBELLAW type=write size=195809 2024-03-04T17:50:42.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.137+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~OKBSK09J1WW9.0 size=1048576 2024-03-04T17:50:42.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.145+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~OKBSK09J1WW9.1 size=641459 2024-03-04T17:50:42.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.145+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=OKBSK09J1WW9 tag=794TDD6WIGDO type=write size=2738611 2024-03-04T17:50:42.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.149+0000 7fda08af98c0 1 < completed write operation key=C4B45GL40IMZ tag=FP3W4VBXQ8ZJ type=write size=242875 2024-03-04T17:50:42.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.153+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=KBZAQ1QX0V0L type=del size=1978432 2024-03-04T17:50:42.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.161+0000 7fda08af98c0 1 < suggested remove operation key=FMPZ2548LBBF tag=AJN6II6R73C8 type=del 2024-03-04T17:50:42.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.169+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~K0B01K52E7IK.0 size=1048576 2024-03-04T17:50:42.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.169+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~K0B01K52E7IK.1 size=1048576 2024-03-04T17:50:42.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.173+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~K0B01K52E7IK.2 size=575034 2024-03-04T17:50:42.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.173+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=K0B01K52E7IK tag=Q7GMWNEJ5T7J type=write size=3720762 2024-03-04T17:50:42.176 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.177+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~0ZTLB0TVA1OT.0 size=1048576 2024-03-04T17:50:42.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.181+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~0ZTLB0TVA1OT.1 size=1048576 2024-03-04T17:50:42.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.185+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~0ZTLB0TVA1OT.2 size=269725 2024-03-04T17:50:42.189 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.189+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=0ZTLB0TVA1OT tag=Q286I806HSC8 type=write size=3415453 2024-03-04T17:50:42.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.193+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=WRJMWZZP1IUJ type=del size=1050501 2024-03-04T17:50:42.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.197+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~C7JQI34UB0YL.0 size=1048576 2024-03-04T17:50:42.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.197+0000 7fda08af98c0 1 < canceled part key=_multipart_DJCCQ6S0E7UG.2~C7JQI34UB0YL.1 size=1048576 2024-03-04T17:50:42.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.201+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~C7JQI34UB0YL.2 size=638071 2024-03-04T17:50:42.206 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.209+0000 7fda08af98c0 1 < canceled multipart upload key=DJCCQ6S0E7UG upload=C7JQI34UB0YL tag=EC4CKKQ4JB1G type=write size=3783799 2024-03-04T17:50:42.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.213+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=WT68XL3GCVQX type=del size=3523747 2024-03-04T17:50:42.214 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.217+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=XAQ2S1K4NKY9 type=del 2024-03-04T17:50:42.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.221+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=UQU1JORK6BJF type=del size=2592270 2024-03-04T17:50:42.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.229+0000 7fda08af98c0 1 < suggested update operation key=1OZKPHMPV3OQ tag=B3ZWKP5A66MD type=del size=1951659 2024-03-04T17:50:42.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.233+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=B66U66YT56K0 type=del size=1216535 2024-03-04T17:50:42.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.241+0000 7fda08af98c0 1 < suggested remove operation key=L0FVKWD7EQRW tag=SP0ROP14CDB2 type=del 2024-03-04T17:50:42.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.245+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=AEFL2RIAVAQG type=write size=358769 2024-03-04T17:50:42.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.249+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=27FLW5443WK3 type=del 2024-03-04T17:50:42.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.253+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=VLSIWNVV7JHW type=del size=1786176 2024-03-04T17:50:42.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.257+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~PH2IRUDYYY5X.0 size=1048576 2024-03-04T17:50:42.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.257+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~PH2IRUDYYY5X.1 size=1048576 2024-03-04T17:50:42.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.257+0000 7fda08af98c0 1 < canceled part key=_multipart_DJCCQ6S0E7UG.2~PH2IRUDYYY5X.2 size=942135 2024-03-04T17:50:42.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.265+0000 7fda08af98c0 1 < canceled multipart upload key=DJCCQ6S0E7UG upload=PH2IRUDYYY5X tag=U13LHO9I1KY9 type=write size=4087863 2024-03-04T17:50:42.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.269+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=940JUXHFHXIY type=del size=1011382 2024-03-04T17:50:42.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.273+0000 7fda08af98c0 1 < completed write operation key=2PA8JDZUOLO5 tag=R2AE1ABKRW64 type=write size=614873 2024-03-04T17:50:42.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.277+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=5U4SM8XX34UO type=write size=670364 2024-03-04T17:50:42.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.281+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~01BX8J69XWGU.0 size=1048576 2024-03-04T17:50:42.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.285+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~01BX8J69XWGU.1 size=1048576 2024-03-04T17:50:42.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.285+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~01BX8J69XWGU.2 size=796262 2024-03-04T17:50:42.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.293+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=01BX8J69XWGU tag=NTEXQAZRBACR type=write size=3941990 2024-03-04T17:50:42.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.301+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~X5RM9GLNKPEX.0 size=1048576 2024-03-04T17:50:42.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.301+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~X5RM9GLNKPEX.1 size=1048576 2024-03-04T17:50:42.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.305+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~X5RM9GLNKPEX.2 size=792066 2024-03-04T17:50:42.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.305+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=X5RM9GLNKPEX tag=CK31CSDEELCB type=write size=3937794 2024-03-04T17:50:42.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.309+0000 7fda08af98c0 1 < completed write operation key=0IGSMFORCSL9 tag=AXHV1F03PFP0 type=write size=96160 2024-03-04T17:50:42.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.317+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~3JFFVFXUO1J1.0 size=1048576 2024-03-04T17:50:42.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.317+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~3JFFVFXUO1J1.1 size=1048576 2024-03-04T17:50:42.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.321+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~3JFFVFXUO1J1.2 size=1008210 2024-03-04T17:50:42.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.321+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=3JFFVFXUO1J1 tag=4PIQ6I8HEK1V type=write size=4153938 2024-03-04T17:50:42.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.325+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=XKS4BMMGZYKN type=del 2024-03-04T17:50:42.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.329+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=J72MQS2V9TXH type=del size=3029947 2024-03-04T17:50:42.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.333+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=8HV1TBR4AK2P type=del 2024-03-04T17:50:42.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.337+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~GNXCKCCWCDCC.0 size=1048576 2024-03-04T17:50:42.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.341+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~GNXCKCCWCDCC.1 size=1048576 2024-03-04T17:50:42.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.345+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~GNXCKCCWCDCC.2 size=372846 2024-03-04T17:50:42.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.345+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=GNXCKCCWCDCC tag=9HBPRVNEZJJG type=write size=3518574 2024-03-04T17:50:42.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.349+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=DYKTK2PAPTYC type=del 2024-03-04T17:50:42.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.353+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~4YKJ4U8UT41M.0 size=1048576 2024-03-04T17:50:42.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.357+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~4YKJ4U8UT41M.1 size=1048576 2024-03-04T17:50:42.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.361+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~4YKJ4U8UT41M.2 size=306630 2024-03-04T17:50:42.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.361+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=4YKJ4U8UT41M tag=SVUY5C01SVKG type=write size=3452358 2024-03-04T17:50:42.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.365+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~OKBSK09J1WW9.0 size=1048576 2024-03-04T17:50:42.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.365+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~OKBSK09J1WW9.1 size=641459 2024-03-04T17:50:42.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.373+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=OKBSK09J1WW9 tag=794TDD6WIGDO type=write size=2738611 2024-03-04T17:50:42.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.377+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=7A400V3CCPC1 type=del size=2767447 2024-03-04T17:50:42.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.381+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=KBZAQ1QX0V0L type=del 2024-03-04T17:50:42.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.385+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=KT5C7OQ6T17R type=del size=3604122 2024-03-04T17:50:42.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.389+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~K0B01K52E7IK.0 size=1048576 2024-03-04T17:50:42.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.393+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~K0B01K52E7IK.1 size=1048576 2024-03-04T17:50:42.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.397+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~K0B01K52E7IK.2 size=575034 2024-03-04T17:50:42.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.401+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=K0B01K52E7IK tag=Q7GMWNEJ5T7J type=write size=3720762 2024-03-04T17:50:42.404 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.405+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=SZAF8T4W5XX9 type=del size=412236 2024-03-04T17:50:42.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.409+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=WRJMWZZP1IUJ type=del 2024-03-04T17:50:42.412 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.413+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=IOQ4BJDU2VXV type=del size=1094932 2024-03-04T17:50:42.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.417+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=WT68XL3GCVQX type=del 2024-03-04T17:50:42.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.421+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=FJPNE5P8G6O2 type=del size=2671474 2024-03-04T17:50:42.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.425+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=UQU1JORK6BJF type=del 2024-03-04T17:50:42.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.433+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=CKLQ02VGPLKX type=del size=47582 2024-03-04T17:50:42.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.441+0000 7fda08af98c0 1 < suggested remove operation key=09KBCA2XURTF tag=B66U66YT56K0 type=del 2024-03-04T17:50:42.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.449+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~1IF532ZSQ7RU.0 size=1048576 2024-03-04T17:50:42.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.449+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~1IF532ZSQ7RU.1 size=905196 2024-03-04T17:50:42.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.449+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=1IF532ZSQ7RU tag=0MBI42PPT2RL type=write size=3002348 2024-03-04T17:50:42.455 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.457+0000 7fda08af98c0 1 < suggested remove operation key=0IGSMFORCSL9 tag=AEFL2RIAVAQG type=write 2024-03-04T17:50:42.459 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.461+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=AP0AESNKW3EV type=del size=935272 2024-03-04T17:50:42.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.465+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=VLSIWNVV7JHW type=del 2024-03-04T17:50:42.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.469+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=DE2Q81SXMZJW type=write size=735686 2024-03-04T17:50:42.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.473+0000 7fda08af98c0 1 < suggested remove operation key=HCA7QLGM2O4O tag=940JUXHFHXIY type=del 2024-03-04T17:50:42.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.477+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=WXLP4GEK35QM type=del size=1644006 2024-03-04T17:50:42.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.481+0000 7fda08af98c0 1 < completed write operation key=IJSYAVR5SWPS tag=5U4SM8XX34UO type=write size=670364 2024-03-04T17:50:42.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.493+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~476NPC41JZ56.0 size=1048576 2024-03-04T17:50:42.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.493+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~476NPC41JZ56.1 size=720319 2024-03-04T17:50:42.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.493+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=476NPC41JZ56 tag=8XG15KWBQPJU type=write size=2817471 2024-03-04T17:50:42.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.497+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~X5RM9GLNKPEX.0 size=1048576 2024-03-04T17:50:42.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.497+0000 7fda08af98c0 1 < canceled part key=_multipart_C1GK00FM2ZYA.2~X5RM9GLNKPEX.1 size=1048576 2024-03-04T17:50:42.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.505+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~X5RM9GLNKPEX.2 size=792066 2024-03-04T17:50:42.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.513+0000 7fda08af98c0 1 < canceled multipart upload key=C1GK00FM2ZYA upload=X5RM9GLNKPEX tag=CK31CSDEELCB type=write size=3937794 2024-03-04T17:50:42.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.521+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~H5ZJXB4T6327.0 size=1048576 2024-03-04T17:50:42.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.521+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~H5ZJXB4T6327.1 size=196187 2024-03-04T17:50:42.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.521+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=H5ZJXB4T6327 tag=MRQG3K3TZHE3 type=write size=2293339 2024-03-04T17:50:42.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.525+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~3JFFVFXUO1J1.0 size=1048576 2024-03-04T17:50:42.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.529+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~3JFFVFXUO1J1.1 size=1048576 2024-03-04T17:50:42.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.533+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~3JFFVFXUO1J1.2 size=1008210 2024-03-04T17:50:42.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.541+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=3JFFVFXUO1J1 tag=4PIQ6I8HEK1V type=write size=4153938 2024-03-04T17:50:42.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.545+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=MB16RCIRKUHQ type=del size=4063232 2024-03-04T17:50:42.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.549+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=J72MQS2V9TXH type=del 2024-03-04T17:50:42.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.553+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=RP17OGHJUXE3 type=del size=523073 2024-03-04T17:50:42.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.557+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~GNXCKCCWCDCC.0 size=1048576 2024-03-04T17:50:42.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.557+0000 7fda08af98c0 1 < canceled part key=_multipart_HCA7QLGM2O4O.2~GNXCKCCWCDCC.1 size=1048576 2024-03-04T17:50:42.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.561+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~GNXCKCCWCDCC.2 size=372846 2024-03-04T17:50:42.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.565+0000 7fda08af98c0 1 < canceled multipart upload key=HCA7QLGM2O4O upload=GNXCKCCWCDCC tag=9HBPRVNEZJJG type=write size=3518574 2024-03-04T17:50:42.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.569+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=EPXOF1H19L7M type=del size=2566376 2024-03-04T17:50:42.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.573+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~4YKJ4U8UT41M.0 size=1048576 2024-03-04T17:50:42.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.577+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~4YKJ4U8UT41M.1 size=1048576 2024-03-04T17:50:42.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.577+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~4YKJ4U8UT41M.2 size=306630 2024-03-04T17:50:42.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.585+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=4YKJ4U8UT41M tag=SVUY5C01SVKG type=write size=3452358 2024-03-04T17:50:42.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.593+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~AI7S9N9SFJS5.0 size=532561 2024-03-04T17:50:42.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.593+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=AI7S9N9SFJS5 tag=6SHUKRKBAT9D type=write size=1581137 2024-03-04T17:50:42.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.597+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=7A400V3CCPC1 type=del 2024-03-04T17:50:42.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.601+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=6HWNR3WW1LMP type=del size=4068424 2024-03-04T17:50:42.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.605+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=KT5C7OQ6T17R type=del 2024-03-04T17:50:42.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.613+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~UD8YTI85M0F0.0 size=1048576 2024-03-04T17:50:42.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.613+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~UD8YTI85M0F0.1 size=847013 2024-03-04T17:50:42.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.613+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=UD8YTI85M0F0 tag=R9IWH4L9W81A type=write size=2944165 2024-03-04T17:50:42.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.617+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=SZAF8T4W5XX9 type=del 2024-03-04T17:50:42.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.621+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=RAKM4S7XY0YE type=del size=466127 2024-03-04T17:50:42.627 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.629+0000 7fda08af98c0 1 < suggested update operation key=BYQFZSJ3WH07 tag=IOQ4BJDU2VXV type=del size=1094932 2024-03-04T17:50:42.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.633+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=L4T0AF923RMT type=del size=3980025 2024-03-04T17:50:42.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.641+0000 7fda08af98c0 1 < suggested remove operation key=ITIVF3UAT9NK tag=FJPNE5P8G6O2 type=del 2024-03-04T17:50:42.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.649+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~5P6DWRSDK0WI.0 size=1048576 2024-03-04T17:50:42.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.649+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~5P6DWRSDK0WI.1 size=660312 2024-03-04T17:50:42.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.649+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=5P6DWRSDK0WI tag=KZ7VKTNHA1DP type=write size=2757464 2024-03-04T17:50:42.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.653+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=CKLQ02VGPLKX type=del 2024-03-04T17:50:42.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.661+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~MLWV2WKKCFIT.0 size=245703 2024-03-04T17:50:42.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.661+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=MLWV2WKKCFIT tag=VUWD0TM6M1S1 type=write size=1294279 2024-03-04T17:50:42.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.665+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~1IF532ZSQ7RU.0 size=1048576 2024-03-04T17:50:42.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.669+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~1IF532ZSQ7RU.1 size=905196 2024-03-04T17:50:42.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.673+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=1IF532ZSQ7RU tag=0MBI42PPT2RL type=write size=3002348 2024-03-04T17:50:42.676 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.677+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=E7D9DBUHYBCL type=del size=3802090 2024-03-04T17:50:42.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.681+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=AP0AESNKW3EV type=del 2024-03-04T17:50:42.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.689+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~PGT8BI1U67UB.0 size=1048576 2024-03-04T17:50:42.690 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.693+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~PGT8BI1U67UB.1 size=1048576 2024-03-04T17:50:42.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.693+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~PGT8BI1U67UB.2 size=602909 2024-03-04T17:50:42.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.693+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=PGT8BI1U67UB tag=2QIO2T4H116E type=write size=3748637 2024-03-04T17:50:42.697 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.701+0000 7fda08af98c0 1 < completed write operation key=TOFWDFH0HL3Q tag=DE2Q81SXMZJW type=write size=735686 2024-03-04T17:50:42.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.705+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=GMGJ1C5EHICH type=del size=818828 2024-03-04T17:50:42.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.709+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=WXLP4GEK35QM type=del 2024-03-04T17:50:42.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.713+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=CP2JIIMR8OI3 type=del size=3126805 2024-03-04T17:50:42.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.717+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~476NPC41JZ56.0 size=1048576 2024-03-04T17:50:42.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.721+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~476NPC41JZ56.1 size=720319 2024-03-04T17:50:42.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.725+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=476NPC41JZ56 tag=8XG15KWBQPJU type=write size=2817471 2024-03-04T17:50:42.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.733+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~7IQ08250YQ3M.0 size=1048576 2024-03-04T17:50:42.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.737+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~7IQ08250YQ3M.1 size=1048576 2024-03-04T17:50:42.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.741+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~7IQ08250YQ3M.2 size=534886 2024-03-04T17:50:42.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.741+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=7IQ08250YQ3M tag=YHDH0DZHOY2W type=write size=3680614 2024-03-04T17:50:42.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.741+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~H5ZJXB4T6327.0 size=1048576 2024-03-04T17:50:42.744 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.745+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~H5ZJXB4T6327.1 size=196187 2024-03-04T17:50:42.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.753+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=H5ZJXB4T6327 tag=MRQG3K3TZHE3 type=write size=2293339 2024-03-04T17:50:42.754 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.757+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=WZQTHXTQDRZ1 type=del size=2685251 2024-03-04T17:50:42.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.761+0000 7fda08af98c0 1 < canceled operation key=M5BMX7B7RZYL tag=MB16RCIRKUHQ type=del size=4063232 2024-03-04T17:50:42.762 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.765+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=NI7566WMU6PB type=write size=982470 2024-03-04T17:50:42.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.773+0000 7fda08af98c0 1 < suggested update operation key=0IGSMFORCSL9 tag=RP17OGHJUXE3 type=del size=523073 2024-03-04T17:50:42.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.777+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=W7PYXL6DVKNE type=del size=18917 2024-03-04T17:50:42.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.781+0000 7fda08af98c0 -1 < failed to complete operation key=0IGSMFORCSL9 tag=EPXOF1H19L7M type=del size=2566376: (22) Invalid argument 2024-03-04T17:50:42.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.785+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=1AAQQN5DIHWT type=write size=678153 2024-03-04T17:50:42.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.785+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~AI7S9N9SFJS5.0 size=532561 2024-03-04T17:50:42.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.793+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=AI7S9N9SFJS5 tag=6SHUKRKBAT9D type=write size=1581137 2024-03-04T17:50:42.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.797+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=V0I7YX961LND type=del size=2667607 2024-03-04T17:50:42.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.801+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=6HWNR3WW1LMP type=del 2024-03-04T17:50:42.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.805+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=SJGT71CXJVDE type=del size=4066164 2024-03-04T17:50:42.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.809+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~UD8YTI85M0F0.0 size=1048576 2024-03-04T17:50:42.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.813+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~UD8YTI85M0F0.1 size=847013 2024-03-04T17:50:42.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.817+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=UD8YTI85M0F0 tag=R9IWH4L9W81A type=write size=2944165 2024-03-04T17:50:42.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.825+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=AJ1X8N53XWBG type=del size=2184989 2024-03-04T17:50:42.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.829+0000 7fda08af98c0 -1 < failed to complete operation key=0IGSMFORCSL9 tag=RAKM4S7XY0YE type=del size=466127: (22) Invalid argument 2024-03-04T17:50:42.830 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.833+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=DOO9SE7KMVIC type=del size=4062511 2024-03-04T17:50:42.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.837+0000 7fda08af98c0 1 < canceled operation key=Z2JQBN84QQIB tag=L4T0AF923RMT type=del size=3980025 2024-03-04T17:50:42.838 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.841+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=9IYMECZC06W4 type=del size=707008 2024-03-04T17:50:42.842 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.845+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~5P6DWRSDK0WI.0 size=1048576 2024-03-04T17:50:42.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.845+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~5P6DWRSDK0WI.1 size=660312 2024-03-04T17:50:42.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.853+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=5P6DWRSDK0WI tag=KZ7VKTNHA1DP type=write size=2757464 2024-03-04T17:50:42.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.857+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=HKKLOPXTTW89 type=del size=1696976 2024-03-04T17:50:42.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.861+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~MLWV2WKKCFIT.0 size=245703 2024-03-04T17:50:42.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.873+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=MLWV2WKKCFIT tag=VUWD0TM6M1S1 type=write size=1294279 2024-03-04T17:50:42.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.877+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=R750M10L883Q type=del size=2729711 2024-03-04T17:50:42.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.881+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=E7D9DBUHYBCL type=del 2024-03-04T17:50:42.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.885+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~WX502A311NHU.0 size=32676 2024-03-04T17:50:42.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.885+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=WX502A311NHU tag=Q4HBR3BW6RRG type=write size=1081252 2024-03-04T17:50:42.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.889+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~PGT8BI1U67UB.0 size=1048576 2024-03-04T17:50:42.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.893+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~PGT8BI1U67UB.1 size=1048576 2024-03-04T17:50:42.894 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.897+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~PGT8BI1U67UB.2 size=602909 2024-03-04T17:50:42.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.901+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=PGT8BI1U67UB tag=2QIO2T4H116E type=write size=3748637 2024-03-04T17:50:42.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.909+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~K3NCEUOPFDVK.0 size=1048576 2024-03-04T17:50:42.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.913+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~K3NCEUOPFDVK.1 size=1048576 2024-03-04T17:50:42.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.917+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~K3NCEUOPFDVK.2 size=757787 2024-03-04T17:50:42.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.917+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=K3NCEUOPFDVK tag=N8FL4P9VY0L8 type=write size=3903515 2024-03-04T17:50:42.923 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.925+0000 7fda08af98c0 1 < suggested remove operation key=L0FVKWD7EQRW tag=GMGJ1C5EHICH type=del 2024-03-04T17:50:42.927 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.929+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=R1BDKCT7DDR4 type=del size=2677051 2024-03-04T17:50:42.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.937+0000 7fda08af98c0 1 < suggested remove operation key=ITIVF3UAT9NK tag=CP2JIIMR8OI3 type=del 2024-03-04T17:50:42.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.941+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=CAAZTCLM041O type=del size=3884739 2024-03-04T17:50:42.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.945+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~7IQ08250YQ3M.0 size=1048576 2024-03-04T17:50:42.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.945+0000 7fda08af98c0 1 < canceled part key=_multipart_VJVORUG95FLC.2~7IQ08250YQ3M.1 size=1048576 2024-03-04T17:50:42.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.945+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~7IQ08250YQ3M.2 size=534886 2024-03-04T17:50:42.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.953+0000 7fda08af98c0 1 < canceled multipart upload key=VJVORUG95FLC upload=7IQ08250YQ3M tag=YHDH0DZHOY2W type=write size=3680614 2024-03-04T17:50:42.955 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.957+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=26AW9J9NRDK6 type=del size=3123227 2024-03-04T17:50:42.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.961+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=WZQTHXTQDRZ1 type=del 2024-03-04T17:50:42.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.969+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~XK4AS9UX565U.0 size=582555 2024-03-04T17:50:42.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.969+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=XK4AS9UX565U tag=DX73ZIWC1R3N type=write size=1631131 2024-03-04T17:50:42.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.969+0000 7fda08af98c0 1 < completed write operation key=Z3J7W9V8HQKH tag=NI7566WMU6PB type=write size=982470 2024-03-04T17:50:42.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.973+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=Q635TJFU0X7M type=del size=96074 2024-03-04T17:50:42.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.981+0000 7fda08af98c0 1 < suggested remove operation key=IJSYAVR5SWPS tag=W7PYXL6DVKNE type=del 2024-03-04T17:50:42.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.985+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=WMXRWO1UX1VZ type=del size=1878223 2024-03-04T17:50:42.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.989+0000 7fda08af98c0 1 < completed write operation key=GDGTJZ21TG32 tag=1AAQQN5DIHWT type=write size=678153 2024-03-04T17:50:42.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:42.993+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=YOIQRIUXEHK3 type=del size=605673 2024-03-04T17:50:43.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.001+0000 7fda08af98c0 1 < suggested remove operation key=RXVSQVVNXVF0 tag=V0I7YX961LND type=del 2024-03-04T17:50:43.003 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.005+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=86KCBNUSZCX7 type=del size=1538317 2024-03-04T17:50:43.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.013+0000 7fda08af98c0 1 < suggested update operation key=X5MEJ7G8X9HE tag=SJGT71CXJVDE type=del size=4066164 2024-03-04T17:50:43.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.017+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=P0T3MN3C3ZSH type=del size=3041745 2024-03-04T17:50:43.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.021+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=AJ1X8N53XWBG type=del 2024-03-04T17:50:43.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.025+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=QD43FQRBDLF4 type=del size=2719386 2024-03-04T17:50:43.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.029+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=DOO9SE7KMVIC type=del 2024-03-04T17:50:43.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.033+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=9EHK9K2TKCLP type=del size=3807578 2024-03-04T17:50:43.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.037+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=9IYMECZC06W4 type=del 2024-03-04T17:50:43.039 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.041+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=4REKJHNRJAC6 type=del size=3214676 2024-03-04T17:50:43.042 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.045+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=HKKLOPXTTW89 type=del 2024-03-04T17:50:43.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.049+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=KU1I7V8AW81G type=write size=887482 2024-03-04T17:50:43.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.057+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=R750M10L883Q type=del 2024-03-04T17:50:43.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.061+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=N550XV7HKF0E type=del size=2197548 2024-03-04T17:50:43.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.065+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~WX502A311NHU.0 size=32676 2024-03-04T17:50:43.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.069+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=WX502A311NHU tag=Q4HBR3BW6RRG type=write size=1081252 2024-03-04T17:50:43.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.073+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=4JBYQ8B70JB2 type=write size=220141 2024-03-04T17:50:43.077 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.077+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~K3NCEUOPFDVK.0 size=1048576 2024-03-04T17:50:43.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.081+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~K3NCEUOPFDVK.1 size=1048576 2024-03-04T17:50:43.084 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.085+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~K3NCEUOPFDVK.2 size=757787 2024-03-04T17:50:43.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.093+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=K3NCEUOPFDVK tag=N8FL4P9VY0L8 type=write size=3903515 2024-03-04T17:50:43.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.097+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~0LW0E1A1N11A.0 size=693024 2024-03-04T17:50:43.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.097+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=0LW0E1A1N11A tag=MTAQQB3O3UUF type=write size=1741600 2024-03-04T17:50:43.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.105+0000 7fda08af98c0 1 < suggested update operation key=AO3QS6CM7I5D tag=R1BDKCT7DDR4 type=del size=2677051 2024-03-04T17:50:43.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.109+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=PLYSVK3NWUYR type=del size=1893082 2024-03-04T17:50:43.111 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.113+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=CAAZTCLM041O type=del 2024-03-04T17:50:43.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.121+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~L1U1GZ11SLEG.0 size=1048576 2024-03-04T17:50:43.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.121+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~L1U1GZ11SLEG.1 size=816260 2024-03-04T17:50:43.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.121+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=L1U1GZ11SLEG tag=825UW5C1D8U1 type=write size=2913412 2024-03-04T17:50:43.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.125+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=26AW9J9NRDK6 type=del 2024-03-04T17:50:43.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.129+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=ZES5LC9KZ9H7 type=write size=420242 2024-03-04T17:50:43.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.133+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~XK4AS9UX565U.0 size=582555 2024-03-04T17:50:43.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.141+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=XK4AS9UX565U tag=DX73ZIWC1R3N type=write size=1631131 2024-03-04T17:50:43.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.145+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=962L791WE0PW type=write size=200374 2024-03-04T17:50:43.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.149+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=Q635TJFU0X7M type=del 2024-03-04T17:50:43.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.153+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=5FDBSYQADGFI type=write size=551214 2024-03-04T17:50:43.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.157+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=WMXRWO1UX1VZ type=del 2024-03-04T17:50:43.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.161+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=FQ0RS0PJEE4E type=write size=841460 2024-03-04T17:50:43.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.169+0000 7fda08af98c0 1 < suggested remove operation key=JA4JK99I0942 tag=YOIQRIUXEHK3 type=del 2024-03-04T17:50:43.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.169+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=MCCCYLKNHUMG type=write size=510331 2024-03-04T17:50:43.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.177+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=86KCBNUSZCX7 type=del 2024-03-04T17:50:43.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.181+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=SYFP8TFWA2JC type=del size=4053 2024-03-04T17:50:43.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.189+0000 7fda08af98c0 1 < suggested remove operation key=8YL7C9PI58TK tag=P0T3MN3C3ZSH type=del 2024-03-04T17:50:43.190 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.193+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=PPJXDFCFPNDP type=del size=3895792 2024-03-04T17:50:43.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.197+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=QD43FQRBDLF4 type=del 2024-03-04T17:50:43.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.201+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=LX0GW0WSGN4A type=del size=4097495 2024-03-04T17:50:43.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.201+0000 7fda08af98c0 1 < canceled operation key=L0FVKWD7EQRW tag=9EHK9K2TKCLP type=del size=3807578 2024-03-04T17:50:43.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.209+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~8NZ6MCITDUM7.0 size=1048576 2024-03-04T17:50:43.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.213+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~8NZ6MCITDUM7.1 size=170310 2024-03-04T17:50:43.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.213+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=8NZ6MCITDUM7 tag=Q4GZNW66YZP0 type=write size=2267462 2024-03-04T17:50:43.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.221+0000 7fda08af98c0 1 < suggested remove operation key=C4B45GL40IMZ tag=4REKJHNRJAC6 type=del 2024-03-04T17:50:43.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.225+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=TZX2OT002CUT type=del size=1919706 2024-03-04T17:50:43.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.229+0000 7fda08af98c0 1 < completed write operation key=Z2JQBN84QQIB tag=KU1I7V8AW81G type=write size=887482 2024-03-04T17:50:43.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.237+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=5JD756VEDKO6 type=write size=319424 2024-03-04T17:50:43.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.245+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=N550XV7HKF0E type=del 2024-03-04T17:50:43.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.249+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=Y7RKXOABCEZA type=del size=612808 2024-03-04T17:50:43.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.253+0000 7fda08af98c0 1 < completed write operation key=IJSYAVR5SWPS tag=4JBYQ8B70JB2 type=write size=220141 2024-03-04T17:50:43.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.257+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=2WW6KG5X3Y2K type=write size=75429 2024-03-04T17:50:43.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.261+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~0LW0E1A1N11A.0 size=693024 2024-03-04T17:50:43.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.265+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=0LW0E1A1N11A tag=MTAQQB3O3UUF type=write size=1741600 2024-03-04T17:50:43.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.273+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~DP2Z6FFJ9MT0.0 size=155718 2024-03-04T17:50:43.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.273+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=DP2Z6FFJ9MT0 tag=5W48ANH6HI2G type=write size=1204294 2024-03-04T17:50:43.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.281+0000 7fda08af98c0 1 < suggested remove operation key=FMPZ2548LBBF tag=PLYSVK3NWUYR type=del 2024-03-04T17:50:43.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.285+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=YCLHONJUY89N type=write size=88783 2024-03-04T17:50:43.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.285+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~L1U1GZ11SLEG.0 size=1048576 2024-03-04T17:50:43.287 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.289+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~L1U1GZ11SLEG.1 size=816260 2024-03-04T17:50:43.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.293+0000 7fda08af98c0 1 < canceled multipart upload key=JA4JK99I0942 upload=L1U1GZ11SLEG tag=825UW5C1D8U1 type=write size=2913412 2024-03-04T17:50:43.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.297+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=KNH429BWGTMM type=del size=2086737 2024-03-04T17:50:43.299 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.301+0000 7fda08af98c0 1 < completed write operation key=GDGTJZ21TG32 tag=ZES5LC9KZ9H7 type=write size=420242 2024-03-04T17:50:43.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.305+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=4LGQTH6JYXWA type=del size=3305434 2024-03-04T17:50:43.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.309+0000 7fda08af98c0 -1 < failed to complete operation key=C4B45GL40IMZ tag=962L791WE0PW type=write size=200374: (22) Invalid argument 2024-03-04T17:50:43.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.313+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~DBK9JHWOGWQN.0 size=1048576 2024-03-04T17:50:43.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.317+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~DBK9JHWOGWQN.1 size=481881 2024-03-04T17:50:43.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.317+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=DBK9JHWOGWQN tag=V1HE0LKNHSLF type=write size=2579033 2024-03-04T17:50:43.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.325+0000 7fda08af98c0 1 < suggested remove operation key=X5MEJ7G8X9HE tag=5FDBSYQADGFI type=write 2024-03-04T17:50:43.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.333+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~WEIQJHUQMI0G.0 size=1048576 2024-03-04T17:50:43.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.333+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~WEIQJHUQMI0G.1 size=867477 2024-03-04T17:50:43.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.333+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=WEIQJHUQMI0G tag=RS06UTZ01LUZ type=write size=2964629 2024-03-04T17:50:43.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.337+0000 7fda08af98c0 1 < completed write operation key=ITIVF3UAT9NK tag=FQ0RS0PJEE4E type=write size=841460 2024-03-04T17:50:43.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.345+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~L2NCS3NQKNV4.0 size=373668 2024-03-04T17:50:43.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.345+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=L2NCS3NQKNV4 tag=TVQNQAXWHY19 type=write size=1422244 2024-03-04T17:50:43.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.349+0000 7fda08af98c0 1 < completed write operation key=M5BMX7B7RZYL tag=MCCCYLKNHUMG type=write size=510331 2024-03-04T17:50:43.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.353+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~KZ7P82NCK16U.0 size=674729 2024-03-04T17:50:43.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.353+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=KZ7P82NCK16U tag=JMVURHTZI7G8 type=write size=1723305 2024-03-04T17:50:43.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.357+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=SYFP8TFWA2JC type=del 2024-03-04T17:50:43.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.365+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~TM8VC3I1EMTA.0 size=1048576 2024-03-04T17:50:43.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.369+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~TM8VC3I1EMTA.1 size=650007 2024-03-04T17:50:43.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.369+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=TM8VC3I1EMTA tag=DE0LGA7UOZC1 type=write size=2747159 2024-03-04T17:50:43.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.373+0000 7fda08af98c0 -1 < failed to complete operation key=C4B45GL40IMZ tag=PPJXDFCFPNDP type=del size=3895792: (22) Invalid argument 2024-03-04T17:50:43.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.377+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=3F8IHDD4BUA7 type=del size=566658 2024-03-04T17:50:43.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.385+0000 7fda08af98c0 1 < suggested remove operation key=DJCCQ6S0E7UG tag=LX0GW0WSGN4A type=del 2024-03-04T17:50:43.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.389+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=WGJNBW03FAOW type=del size=3082031 2024-03-04T17:50:43.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.389+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~8NZ6MCITDUM7.0 size=1048576 2024-03-04T17:50:43.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.393+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~8NZ6MCITDUM7.1 size=170310 2024-03-04T17:50:43.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.401+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=8NZ6MCITDUM7 tag=Q4GZNW66YZP0 type=write size=2267462 2024-03-04T17:50:43.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.405+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=4334BCA4L9V2 type=del size=2232699 2024-03-04T17:50:43.410 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.413+0000 7fda08af98c0 1 < suggested update operation key=AO3QS6CM7I5D tag=TZX2OT002CUT type=del size=1919706 2024-03-04T17:50:43.414 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.417+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=N0VSIHHTEVZT type=del size=828951 2024-03-04T17:50:43.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.425+0000 7fda08af98c0 1 < suggested update operation key=1OZKPHMPV3OQ tag=5JD756VEDKO6 type=write size=319424 2024-03-04T17:50:43.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.429+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~IWZIPDLJUHBH.0 size=1048576 2024-03-04T17:50:43.432 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.433+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~IWZIPDLJUHBH.1 size=1048576 2024-03-04T17:50:43.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.437+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~IWZIPDLJUHBH.2 size=943215 2024-03-04T17:50:43.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.437+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=IWZIPDLJUHBH tag=5FJHD1R5EQ78 type=write size=4088943 2024-03-04T17:50:43.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.441+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=Y7RKXOABCEZA type=del 2024-03-04T17:50:43.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.449+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~8XH0BJ8IX074.0 size=1048576 2024-03-04T17:50:43.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.449+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~8XH0BJ8IX074.1 size=917493 2024-03-04T17:50:43.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.449+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=8XH0BJ8IX074 tag=FYLDK20OPKK9 type=write size=3014645 2024-03-04T17:50:43.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.453+0000 7fda08af98c0 1 < completed write operation key=JA4JK99I0942 tag=2WW6KG5X3Y2K type=write size=75429 2024-03-04T17:50:43.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.457+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=TPE2HOPCWHQ4 type=del size=4188341 2024-03-04T17:50:43.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.461+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~DP2Z6FFJ9MT0.0 size=155718 2024-03-04T17:50:43.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.469+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=DP2Z6FFJ9MT0 tag=5W48ANH6HI2G type=write size=1204294 2024-03-04T17:50:43.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.477+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~CSB3587T3ODE.0 size=1048576 2024-03-04T17:50:43.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.477+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~CSB3587T3ODE.1 size=560656 2024-03-04T17:50:43.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.477+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=CSB3587T3ODE tag=WEX3KW6DOMA0 type=write size=2657808 2024-03-04T17:50:43.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.481+0000 7fda08af98c0 1 < completed write operation key=3UWSL7NJLACY tag=YCLHONJUY89N type=write size=88783 2024-03-04T17:50:43.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.489+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=5S8H8OKWOC4N type=del size=2038937 2024-03-04T17:50:43.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.493+0000 7fda08af98c0 1 < suggested remove operation key=C4B45GL40IMZ tag=KNH429BWGTMM type=del 2024-03-04T17:50:43.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.497+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=631JDYH5UTSH type=del size=4089464 2024-03-04T17:50:43.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.501+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=4LGQTH6JYXWA type=del 2024-03-04T17:50:43.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.509+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~37KAUGVCI03G.0 size=247059 2024-03-04T17:50:43.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.509+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=37KAUGVCI03G tag=L0DY0JOX2Y9U type=write size=1295635 2024-03-04T17:50:43.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.513+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~DBK9JHWOGWQN.0 size=1048576 2024-03-04T17:50:43.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.513+0000 7fda08af98c0 1 < canceled part key=_multipart_VJVORUG95FLC.2~DBK9JHWOGWQN.1 size=481881 2024-03-04T17:50:43.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.521+0000 7fda08af98c0 1 < canceled multipart upload key=VJVORUG95FLC upload=DBK9JHWOGWQN tag=V1HE0LKNHSLF type=write size=2579033 2024-03-04T17:50:43.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.525+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~D6W920V3JGCO.0 size=550750 2024-03-04T17:50:43.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.525+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=D6W920V3JGCO tag=45EXG5U64U5N type=write size=1599326 2024-03-04T17:50:43.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.529+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~WEIQJHUQMI0G.0 size=1048576 2024-03-04T17:50:43.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.533+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~WEIQJHUQMI0G.1 size=867477 2024-03-04T17:50:43.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.541+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=WEIQJHUQMI0G tag=RS06UTZ01LUZ type=write size=2964629 2024-03-04T17:50:43.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.545+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=7M9M6KGCXL06 type=del size=1783844 2024-03-04T17:50:43.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.545+0000 7fda08af98c0 1 < canceled part key=_multipart_VJVORUG95FLC.2~L2NCS3NQKNV4.0 size=373668 2024-03-04T17:50:43.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.549+0000 7fda08af98c0 1 < canceled multipart upload key=VJVORUG95FLC upload=L2NCS3NQKNV4 tag=TVQNQAXWHY19 type=write size=1422244 2024-03-04T17:50:43.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.557+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~INF6A0RGWX5W.0 size=295448 2024-03-04T17:50:43.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.557+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=INF6A0RGWX5W tag=D2N2GIVYEG3S type=write size=1344024 2024-03-04T17:50:43.560 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.561+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~KZ7P82NCK16U.0 size=674729 2024-03-04T17:50:43.565 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.569+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=KZ7P82NCK16U tag=JMVURHTZI7G8 type=write size=1723305 2024-03-04T17:50:43.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.573+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=Z7TNWMWCCUCC type=del size=654019 2024-03-04T17:50:43.573 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.577+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~TM8VC3I1EMTA.0 size=1048576 2024-03-04T17:50:43.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.577+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~TM8VC3I1EMTA.1 size=650007 2024-03-04T17:50:43.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.585+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=TM8VC3I1EMTA tag=DE0LGA7UOZC1 type=write size=2747159 2024-03-04T17:50:43.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.593+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~U0DI3B41BL50.0 size=1048576 2024-03-04T17:50:43.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.601+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~U0DI3B41BL50.1 size=896688 2024-03-04T17:50:43.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.601+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=U0DI3B41BL50 tag=NS31TUK0D47K type=write size=2993840 2024-03-04T17:50:43.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.609+0000 7fda08af98c0 1 < suggested update operation key=HCKJZUMAK0Z0 tag=3F8IHDD4BUA7 type=del size=566658 2024-03-04T17:50:43.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.613+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=YMNEZZSOIMW2 type=del size=2837420 2024-03-04T17:50:43.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.613+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=WGJNBW03FAOW type=del 2024-03-04T17:50:43.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.617+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=BEY7PTX4I69F type=write size=53296 2024-03-04T17:50:43.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.625+0000 7fda08af98c0 1 < suggested update operation key=2PA8JDZUOLO5 tag=4334BCA4L9V2 type=del size=2232699 2024-03-04T17:50:43.628 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.629+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=IDZDDAX0SK65 type=del size=2755682 2024-03-04T17:50:43.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.633+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=N0VSIHHTEVZT type=del 2024-03-04T17:50:43.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.637+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=QKSKENWNVZX5 type=del size=3418494 2024-03-04T17:50:43.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.641+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~IWZIPDLJUHBH.0 size=1048576 2024-03-04T17:50:43.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.645+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~IWZIPDLJUHBH.1 size=1048576 2024-03-04T17:50:43.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.649+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~IWZIPDLJUHBH.2 size=943215 2024-03-04T17:50:43.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.653+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=IWZIPDLJUHBH tag=5FJHD1R5EQ78 type=write size=4088943 2024-03-04T17:50:43.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.657+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=CVU66S1KWG85 type=del size=3805934 2024-03-04T17:50:43.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.661+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~8XH0BJ8IX074.0 size=1048576 2024-03-04T17:50:43.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.665+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~8XH0BJ8IX074.1 size=917493 2024-03-04T17:50:43.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.673+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=8XH0BJ8IX074 tag=FYLDK20OPKK9 type=write size=3014645 2024-03-04T17:50:43.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.677+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=55OJ9431N768 type=del size=3409683 2024-03-04T17:50:43.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.681+0000 7fda08af98c0 1 < suggested remove operation key=0IGSMFORCSL9 tag=TPE2HOPCWHQ4 type=del 2024-03-04T17:50:43.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.689+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~JHTAYANX3AKZ.0 size=1048576 2024-03-04T17:50:43.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.693+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~JHTAYANX3AKZ.1 size=1048576 2024-03-04T17:50:43.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.697+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~JHTAYANX3AKZ.2 size=561530 2024-03-04T17:50:43.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.697+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=JHTAYANX3AKZ tag=UVDRTZFNT15T type=write size=3707258 2024-03-04T17:50:43.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.701+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~CSB3587T3ODE.0 size=1048576 2024-03-04T17:50:43.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.705+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~CSB3587T3ODE.1 size=560656 2024-03-04T17:50:43.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.709+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=CSB3587T3ODE tag=WEX3KW6DOMA0 type=write size=2657808 2024-03-04T17:50:43.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.717+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~ZD790WGEOGCO.0 size=1016604 2024-03-04T17:50:43.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.717+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=ZD790WGEOGCO tag=EUREBQ0OITR8 type=write size=2065180 2024-03-04T17:50:43.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.725+0000 7fda08af98c0 1 < suggested remove operation key=G0BHMSCO0XLF tag=5S8H8OKWOC4N type=del 2024-03-04T17:50:43.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.733+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~S7AHCWI3FN7N.0 size=1048576 2024-03-04T17:50:43.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.737+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~S7AHCWI3FN7N.1 size=818164 2024-03-04T17:50:43.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.737+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=S7AHCWI3FN7N tag=HLVRJ121PPQK type=write size=2915316 2024-03-04T17:50:43.743 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.745+0000 7fda08af98c0 1 < suggested update operation key=JA4JK99I0942 tag=631JDYH5UTSH type=del size=4089464 2024-03-04T17:50:43.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.749+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=EVJO4D4GCWFU type=del size=3515919 2024-03-04T17:50:43.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.753+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~37KAUGVCI03G.0 size=247059 2024-03-04T17:50:43.755 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.757+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=37KAUGVCI03G tag=L0DY0JOX2Y9U type=write size=1295635 2024-03-04T17:50:43.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.761+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=LJGB2YP9HELB type=del size=2933302 2024-03-04T17:50:43.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.761+0000 7fda08af98c0 1 < canceled part key=_multipart_M5BMX7B7RZYL.2~D6W920V3JGCO.0 size=550750 2024-03-04T17:50:43.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.765+0000 7fda08af98c0 1 < canceled multipart upload key=M5BMX7B7RZYL upload=D6W920V3JGCO tag=45EXG5U64U5N type=write size=1599326 2024-03-04T17:50:43.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.769+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=6YIM66SETYNR type=del size=3361607 2024-03-04T17:50:43.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.773+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=7M9M6KGCXL06 type=del 2024-03-04T17:50:43.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.777+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=LUH2UCYAMH4S type=write size=449095 2024-03-04T17:50:43.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.777+0000 7fda08af98c0 1 < canceled part key=_multipart_RXVSQVVNXVF0.2~INF6A0RGWX5W.0 size=295448 2024-03-04T17:50:43.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.781+0000 7fda08af98c0 1 < canceled multipart upload key=RXVSQVVNXVF0 upload=INF6A0RGWX5W tag=D2N2GIVYEG3S type=write size=1344024 2024-03-04T17:50:43.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.785+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=0EMZSPFUMCD7 type=del size=2676896 2024-03-04T17:50:43.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.789+0000 7fda08af98c0 -1 < failed to complete operation key=2PA8JDZUOLO5 tag=Z7TNWMWCCUCC type=del size=654019: (22) Invalid argument 2024-03-04T17:50:43.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.793+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=47VAYEOQTBL8 type=del size=1706920 2024-03-04T17:50:43.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.797+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~U0DI3B41BL50.0 size=1048576 2024-03-04T17:50:43.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.801+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~U0DI3B41BL50.1 size=896688 2024-03-04T17:50:43.805 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.805+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=U0DI3B41BL50 tag=NS31TUK0D47K type=write size=2993840 2024-03-04T17:50:43.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.809+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=97J52TT3I2NC type=del size=1664346 2024-03-04T17:50:43.816 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.817+0000 7fda08af98c0 1 < suggested update operation key=2PA8JDZUOLO5 tag=YMNEZZSOIMW2 type=del size=2837420 2024-03-04T17:50:43.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.821+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=3SGN9W84DLNT type=del size=1380291 2024-03-04T17:50:43.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.829+0000 7fda08af98c0 1 < suggested update operation key=XG5I3SDRE5JM tag=BEY7PTX4I69F type=write size=53296 2024-03-04T17:50:43.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.833+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=MFQCTI0H7520 type=del size=4021750 2024-03-04T17:50:43.837 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.841+0000 7fda08af98c0 1 < canceled operation key=C4B45GL40IMZ tag=IDZDDAX0SK65 type=del size=2755682 2024-03-04T17:50:43.841 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.845+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=QK8GZ960BHN1 type=write size=853063 2024-03-04T17:50:43.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.849+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=QKSKENWNVZX5 type=del 2024-03-04T17:50:43.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.853+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~7HPBD6M7BC8V.0 size=824145 2024-03-04T17:50:43.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.853+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=7HPBD6M7BC8V tag=1LLOIS4XB4TB type=write size=1872721 2024-03-04T17:50:43.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.857+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=CVU66S1KWG85 type=del 2024-03-04T17:50:43.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.861+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=8DDEQKQJFEX1 type=write size=909651 2024-03-04T17:50:43.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.865+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=55OJ9431N768 type=del 2024-03-04T17:50:43.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.869+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=AV6Y0BKZMQ2S type=write size=631704 2024-03-04T17:50:43.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.873+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~JHTAYANX3AKZ.0 size=1048576 2024-03-04T17:50:43.875 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.877+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~JHTAYANX3AKZ.1 size=1048576 2024-03-04T17:50:43.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.881+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~JHTAYANX3AKZ.2 size=561530 2024-03-04T17:50:43.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.885+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=JHTAYANX3AKZ tag=UVDRTZFNT15T type=write size=3707258 2024-03-04T17:50:43.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.893+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~S7DW2J1D3W63.0 size=488321 2024-03-04T17:50:43.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.893+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=S7DW2J1D3W63 tag=1IUL82PSUE9A type=write size=1536897 2024-03-04T17:50:43.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.897+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~ZD790WGEOGCO.0 size=1016604 2024-03-04T17:50:43.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.905+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=ZD790WGEOGCO tag=EUREBQ0OITR8 type=write size=2065180 2024-03-04T17:50:43.905 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.909+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=JF1629XKAEC0 type=del size=2560045 2024-03-04T17:50:43.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.913+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~S7AHCWI3FN7N.0 size=1048576 2024-03-04T17:50:43.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.913+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~S7AHCWI3FN7N.1 size=818164 2024-03-04T17:50:43.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.921+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=S7AHCWI3FN7N tag=HLVRJ121PPQK type=write size=2915316 2024-03-04T17:50:43.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.925+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=TS2I97SV9ZH5 type=del size=2555943 2024-03-04T17:50:43.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.929+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=EVJO4D4GCWFU type=del 2024-03-04T17:50:43.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.929+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=WMPZFYA731B0 type=del size=3045627 2024-03-04T17:50:43.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.937+0000 7fda08af98c0 1 < suggested remove operation key=X5MEJ7G8X9HE tag=LJGB2YP9HELB type=del 2024-03-04T17:50:43.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.941+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=HF2WFMDIZYIU type=del size=3246840 2024-03-04T17:50:43.944 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.945+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=6YIM66SETYNR type=del size=3361607 2024-03-04T17:50:43.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.953+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~0AO3BORNA51M.0 size=235780 2024-03-04T17:50:43.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.953+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=0AO3BORNA51M tag=Z4CO38UPSWLM type=write size=1284356 2024-03-04T17:50:43.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.961+0000 7fda08af98c0 1 < completed write operation key=FNWA2N9U14HO tag=LUH2UCYAMH4S type=write size=449095 2024-03-04T17:50:43.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.965+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=S5GE84EDVE8R type=del size=3130827 2024-03-04T17:50:43.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.969+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=0EMZSPFUMCD7 type=del size=2676896 2024-03-04T17:50:43.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.973+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=7NWI5Q2ERQN2 type=del size=387849 2024-03-04T17:50:43.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.977+0000 7fda08af98c0 1 < suggested update operation key=V1LDB2HXZ7EF tag=47VAYEOQTBL8 type=del size=1706920 2024-03-04T17:50:43.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.981+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=AWXSR1MEU86B type=del size=922625 2024-03-04T17:50:43.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.985+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=97J52TT3I2NC type=del 2024-03-04T17:50:43.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.989+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=QORNCQMBPGFP type=del size=3716322 2024-03-04T17:50:43.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:43.993+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=3SGN9W84DLNT type=del 2024-03-04T17:50:44.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.001+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~WREG1BO0SHIF.0 size=860261 2024-03-04T17:50:44.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.001+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=WREG1BO0SHIF tag=FWPQYXB35P3X type=write size=1908837 2024-03-04T17:50:44.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.005+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=MFQCTI0H7520 type=del 2024-03-04T17:50:44.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.009+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=CZJZ8EFDYWF6 type=del size=1845395 2024-03-04T17:50:44.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.017+0000 7fda08af98c0 1 < suggested remove operation key=DJCCQ6S0E7UG tag=QK8GZ960BHN1 type=write 2024-03-04T17:50:44.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.025+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~BYFJBROSINB8.0 size=1048576 2024-03-04T17:50:44.025 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.029+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~BYFJBROSINB8.1 size=1048576 2024-03-04T17:50:44.028 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.029+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~BYFJBROSINB8.2 size=905017 2024-03-04T17:50:44.028 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.029+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=BYFJBROSINB8 tag=LM3TZHRKPQV4 type=write size=4050745 2024-03-04T17:50:44.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.029+0000 7fda08af98c0 1 < canceled part key=_multipart_09KBCA2XURTF.2~7HPBD6M7BC8V.0 size=824145 2024-03-04T17:50:44.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.037+0000 7fda08af98c0 1 < canceled multipart upload key=09KBCA2XURTF upload=7HPBD6M7BC8V tag=1LLOIS4XB4TB type=write size=1872721 2024-03-04T17:50:44.042 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.045+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~35XT2QFRR9AX.0 size=102027 2024-03-04T17:50:44.042 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.045+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=35XT2QFRR9AX tag=IOFD3R8QZP2N type=write size=1150603 2024-03-04T17:50:44.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.049+0000 7fda08af98c0 -1 < failed to complete operation key=X5MEJ7G8X9HE tag=8DDEQKQJFEX1 type=write size=909651: (22) Invalid argument 2024-03-04T17:50:44.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.053+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=BM0XBE7QIA4S type=write size=610203 2024-03-04T17:50:44.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.057+0000 7fda08af98c0 1 < completed write operation key=C1GK00FM2ZYA tag=AV6Y0BKZMQ2S type=write size=631704 2024-03-04T17:50:44.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.061+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~F0P5SD3A5Q3W.0 size=562810 2024-03-04T17:50:44.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.061+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=F0P5SD3A5Q3W tag=YMGZ4N3MIH5Y type=write size=1611386 2024-03-04T17:50:44.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.065+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~S7DW2J1D3W63.0 size=488321 2024-03-04T17:50:44.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.073+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=S7DW2J1D3W63 tag=1IUL82PSUE9A type=write size=1536897 2024-03-04T17:50:44.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.077+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=ZS21PAHS5RKH type=write size=334420 2024-03-04T17:50:44.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.081+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=JF1629XKAEC0 type=del 2024-03-04T17:50:44.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.085+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=DDEJ10S0KBNM type=del size=918075 2024-03-04T17:50:44.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.089+0000 7fda08af98c0 -1 < failed to complete operation key=X5MEJ7G8X9HE tag=TS2I97SV9ZH5 type=del size=2555943: (22) Invalid argument 2024-03-04T17:50:44.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.089+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=XWYA9IJU7F6G type=del size=1216929 2024-03-04T17:50:44.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.097+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=WMPZFYA731B0 type=del 2024-03-04T17:50:44.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.101+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=UF7R1HTS0FWJ type=del size=2200975 2024-03-04T17:50:44.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.105+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=HF2WFMDIZYIU type=del 2024-03-04T17:50:44.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.109+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=U3H8AH9O2LPC type=del size=185308 2024-03-04T17:50:44.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.109+0000 7fda08af98c0 1 < canceled part key=_multipart_C4B45GL40IMZ.2~0AO3BORNA51M.0 size=235780 2024-03-04T17:50:44.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.117+0000 7fda08af98c0 1 < canceled multipart upload key=C4B45GL40IMZ upload=0AO3BORNA51M tag=Z4CO38UPSWLM type=write size=1284356 2024-03-04T17:50:44.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.121+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=OSIR7DRNNUES type=del size=1892640 2024-03-04T17:50:44.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.129+0000 7fda08af98c0 1 < suggested update operation key=X5MEJ7G8X9HE tag=S5GE84EDVE8R type=del size=3130827 2024-03-04T17:50:44.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.133+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=BWZ7LDS908P6 type=del size=212186 2024-03-04T17:50:44.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.137+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=7NWI5Q2ERQN2 type=del 2024-03-04T17:50:44.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.141+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=TS2IML423X1L type=del size=2700442 2024-03-04T17:50:44.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.141+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=AWXSR1MEU86B type=del 2024-03-04T17:50:44.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.145+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=80ZJ4158TKGB type=del size=1369236 2024-03-04T17:50:44.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.149+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=QORNCQMBPGFP type=del 2024-03-04T17:50:44.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.153+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=ZBCJW1YWVSXW type=write size=729315 2024-03-04T17:50:44.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.157+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~WREG1BO0SHIF.0 size=860261 2024-03-04T17:50:44.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.165+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=WREG1BO0SHIF tag=FWPQYXB35P3X type=write size=1908837 2024-03-04T17:50:44.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.173+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~1Z31622MRAFL.0 size=21379 2024-03-04T17:50:44.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.173+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=1Z31622MRAFL tag=T7Y37RBNFGRV type=write size=1069955 2024-03-04T17:50:44.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.177+0000 7fda08af98c0 1 < canceled operation key=AO3QS6CM7I5D tag=CZJZ8EFDYWF6 type=del size=1845395 2024-03-04T17:50:44.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.181+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=OTMU5CQMXBL2 type=write size=165975 2024-03-04T17:50:44.183 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.185+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~BYFJBROSINB8.0 size=1048576 2024-03-04T17:50:44.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.189+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~BYFJBROSINB8.1 size=1048576 2024-03-04T17:50:44.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.189+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~BYFJBROSINB8.2 size=905017 2024-03-04T17:50:44.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.197+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=BYFJBROSINB8 tag=LM3TZHRKPQV4 type=write size=4050745 2024-03-04T17:50:44.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.201+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~BS5SCP5XZVAQ.0 size=1048576 2024-03-04T17:50:44.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.205+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~BS5SCP5XZVAQ.1 size=910920 2024-03-04T17:50:44.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.205+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=BS5SCP5XZVAQ tag=DTSN0Z279UDB type=write size=3008072 2024-03-04T17:50:44.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.209+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~35XT2QFRR9AX.0 size=102027 2024-03-04T17:50:44.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.217+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=35XT2QFRR9AX tag=IOFD3R8QZP2N type=write size=1150603 2024-03-04T17:50:44.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.217+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=KZI2XCDT80Z9 type=del size=667414 2024-03-04T17:50:44.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.225+0000 7fda08af98c0 1 < suggested update operation key=JA4JK99I0942 tag=BM0XBE7QIA4S type=write size=610203 2024-03-04T17:50:44.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.233+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~QG9LWXSNHGY0.0 size=1048576 2024-03-04T17:50:44.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.237+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~QG9LWXSNHGY0.1 size=1048576 2024-03-04T17:50:44.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.241+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~QG9LWXSNHGY0.2 size=510323 2024-03-04T17:50:44.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.241+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=QG9LWXSNHGY0 tag=CIN0H43B4J68 type=write size=3656051 2024-03-04T17:50:44.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.245+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~F0P5SD3A5Q3W.0 size=562810 2024-03-04T17:50:44.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.253+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=F0P5SD3A5Q3W tag=YMGZ4N3MIH5Y type=write size=1611386 2024-03-04T17:50:44.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.261+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~9TCH3PZGN3UZ.0 size=44885 2024-03-04T17:50:44.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.261+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=9TCH3PZGN3UZ tag=8SO9QH0EEK3L type=write size=1093461 2024-03-04T17:50:44.262 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.265+0000 7fda08af98c0 1 < canceled operation key=XG2XYBCMUNY5 tag=ZS21PAHS5RKH type=write size=334420 2024-03-04T17:50:44.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.269+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=CY68MGA9C2XR type=del size=2719491 2024-03-04T17:50:44.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.273+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=DDEJ10S0KBNM type=del 2024-03-04T17:50:44.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.277+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=08OV0R89U4U8 type=del size=1627331 2024-03-04T17:50:44.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.285+0000 7fda08af98c0 1 < suggested update operation key=C1GK00FM2ZYA tag=XWYA9IJU7F6G type=del size=1216929 2024-03-04T17:50:44.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.293+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~BX8AXM0EUCK2.0 size=381191 2024-03-04T17:50:44.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.293+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=BX8AXM0EUCK2 tag=9NI2C0WX8CO0 type=write size=1429767 2024-03-04T17:50:44.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.297+0000 7fda08af98c0 1 < canceled operation key=Z2JQBN84QQIB tag=UF7R1HTS0FWJ type=del size=2200975 2024-03-04T17:50:44.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.301+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=XWHLSO8Z56T9 type=del size=3883163 2024-03-04T17:50:44.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.305+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=U3H8AH9O2LPC type=del 2024-03-04T17:50:44.308 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.309+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=RC317T88NT2I type=write size=512499 2024-03-04T17:50:44.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.313+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=OSIR7DRNNUES type=del 2024-03-04T17:50:44.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.317+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=612Y66WNLVDD type=del size=729831 2024-03-04T17:50:44.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.321+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=BWZ7LDS908P6 type=del 2024-03-04T17:50:44.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.325+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=MK20GPOUON03 type=del size=701835 2024-03-04T17:50:44.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.329+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=TS2IML423X1L type=del 2024-03-04T17:50:44.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.337+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=3RPHX8DHGB3S type=del size=3549263 2024-03-04T17:50:44.340 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.341+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=80ZJ4158TKGB type=del 2024-03-04T17:50:44.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.345+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=1QKJ7VEONKBF type=del size=1301245 2024-03-04T17:50:44.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.349+0000 7fda08af98c0 1 < canceled operation key=HCKJZUMAK0Z0 tag=ZBCJW1YWVSXW type=write size=729315 2024-03-04T17:50:44.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.353+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=B2R837WGZ9JY type=del size=1803331 2024-03-04T17:50:44.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.357+0000 7fda08af98c0 1 < canceled part key=_multipart_M5BMX7B7RZYL.2~1Z31622MRAFL.0 size=21379 2024-03-04T17:50:44.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.361+0000 7fda08af98c0 1 < canceled multipart upload key=M5BMX7B7RZYL upload=1Z31622MRAFL tag=T7Y37RBNFGRV type=write size=1069955 2024-03-04T17:50:44.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.369+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~CX3FULBA663O.0 size=444924 2024-03-04T17:50:44.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.369+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=CX3FULBA663O tag=OU39Q4YCGVL4 type=write size=1493500 2024-03-04T17:50:44.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.373+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=OTMU5CQMXBL2 type=write size=165975 2024-03-04T17:50:44.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.373+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=13V7VU7PFXV2 type=del size=1287840 2024-03-04T17:50:44.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.381+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~BS5SCP5XZVAQ.0 size=1048576 2024-03-04T17:50:44.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.381+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~BS5SCP5XZVAQ.1 size=910920 2024-03-04T17:50:44.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.389+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=BS5SCP5XZVAQ tag=DTSN0Z279UDB type=write size=3008072 2024-03-04T17:50:44.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.393+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=5PP1VP8SQV5N type=del size=1772210 2024-03-04T17:50:44.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.397+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=KZI2XCDT80Z9 type=del 2024-03-04T17:50:44.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.401+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=91RH37T7YNBD type=del size=163196 2024-03-04T17:50:44.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.405+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~QG9LWXSNHGY0.0 size=1048576 2024-03-04T17:50:44.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.409+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~QG9LWXSNHGY0.1 size=1048576 2024-03-04T17:50:44.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.413+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~QG9LWXSNHGY0.2 size=510323 2024-03-04T17:50:44.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.417+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=QG9LWXSNHGY0 tag=CIN0H43B4J68 type=write size=3656051 2024-03-04T17:50:44.420 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.421+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=VW0VRBPJ3RSS type=del size=1995806 2024-03-04T17:50:44.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.425+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~9TCH3PZGN3UZ.0 size=44885 2024-03-04T17:50:44.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.429+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=9TCH3PZGN3UZ tag=8SO9QH0EEK3L type=write size=1093461 2024-03-04T17:50:44.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.437+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~CR0WTQMN43A9.0 size=1048576 2024-03-04T17:50:44.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.441+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~CR0WTQMN43A9.1 size=1048576 2024-03-04T17:50:44.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.441+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~CR0WTQMN43A9.2 size=1012555 2024-03-04T17:50:44.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.441+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=CR0WTQMN43A9 tag=MEGKOXUPXIRW type=write size=4158283 2024-03-04T17:50:44.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.449+0000 7fda08af98c0 1 < suggested update operation key=C4B45GL40IMZ tag=CY68MGA9C2XR type=del size=2719491 2024-03-04T17:50:44.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.453+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=FQEM7RROJDP4 type=write size=88212 2024-03-04T17:50:44.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.457+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=08OV0R89U4U8 type=del 2024-03-04T17:50:44.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.461+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=4DEHNI1721Z4 type=del size=2153571 2024-03-04T17:50:44.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.465+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~BX8AXM0EUCK2.0 size=381191 2024-03-04T17:50:44.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.473+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=BX8AXM0EUCK2 tag=9NI2C0WX8CO0 type=write size=1429767 2024-03-04T17:50:44.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.481+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~83K2SU8DJQ6C.0 size=1048576 2024-03-04T17:50:44.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.481+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~83K2SU8DJQ6C.1 size=218873 2024-03-04T17:50:44.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.481+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=83K2SU8DJQ6C tag=BZTF4JPIBKAZ type=write size=2316025 2024-03-04T17:50:44.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.485+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=XWHLSO8Z56T9 type=del 2024-03-04T17:50:44.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.493+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~F9N3OSUL8K0Y.0 size=599007 2024-03-04T17:50:44.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.493+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=F9N3OSUL8K0Y tag=91DW58EDUM9W type=write size=1647583 2024-03-04T17:50:44.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.501+0000 7fda08af98c0 1 < suggested update operation key=HCA7QLGM2O4O tag=RC317T88NT2I type=write size=512499 2024-03-04T17:50:44.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.505+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=RF1Z2FXQ2CMD type=del size=1034242 2024-03-04T17:50:44.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.513+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=612Y66WNLVDD type=del 2024-03-04T17:50:44.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.521+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~CU4XX47LPESU.0 size=300579 2024-03-04T17:50:44.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.521+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=CU4XX47LPESU tag=QILQW0K2ZLTZ type=write size=1349155 2024-03-04T17:50:44.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.525+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=MK20GPOUON03 type=del 2024-03-04T17:50:44.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.529+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~TX03A09TYO59.0 size=895755 2024-03-04T17:50:44.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.529+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=TX03A09TYO59 tag=VRU1QHLK5MZX type=write size=1944331 2024-03-04T17:50:44.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.533+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=3RPHX8DHGB3S type=del 2024-03-04T17:50:44.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.541+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~HO1NYNH1P20V.0 size=1048576 2024-03-04T17:50:44.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.545+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~HO1NYNH1P20V.1 size=4648 2024-03-04T17:50:44.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.545+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=HO1NYNH1P20V tag=FTKCCG4EY6YU type=write size=2101800 2024-03-04T17:50:44.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.549+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=1QKJ7VEONKBF type=del 2024-03-04T17:50:44.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.553+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=R3DJMFASKP8S type=write size=674122 2024-03-04T17:50:44.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.557+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=B2R837WGZ9JY type=del 2024-03-04T17:50:44.559 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.561+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=06GB5Q1KAQCW type=del size=1341668 2024-03-04T17:50:44.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.565+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~CX3FULBA663O.0 size=444924 2024-03-04T17:50:44.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.569+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=CX3FULBA663O tag=OU39Q4YCGVL4 type=write size=1493500 2024-03-04T17:50:44.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.573+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=S46S6627FVM7 type=write size=211458 2024-03-04T17:50:44.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.577+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=13V7VU7PFXV2 type=del 2024-03-04T17:50:44.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.585+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~7RT4MYJI7TAX.0 size=1048576 2024-03-04T17:50:44.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.589+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~7RT4MYJI7TAX.1 size=1048576 2024-03-04T17:50:44.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.589+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~7RT4MYJI7TAX.2 size=218963 2024-03-04T17:50:44.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.589+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=7RT4MYJI7TAX tag=VLUIWDZ8KZ5L type=write size=3364691 2024-03-04T17:50:44.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.593+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=5PP1VP8SQV5N type=del 2024-03-04T17:50:44.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.601+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~HVB8AOUMVV83.0 size=1048576 2024-03-04T17:50:44.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.605+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~HVB8AOUMVV83.1 size=655008 2024-03-04T17:50:44.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.605+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=HVB8AOUMVV83 tag=4HWAXZZNLEN4 type=write size=2752160 2024-03-04T17:50:44.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.609+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=91RH37T7YNBD type=del 2024-03-04T17:50:44.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.617+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~IX25TACJIK17.0 size=1048576 2024-03-04T17:50:44.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.617+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~IX25TACJIK17.1 size=704868 2024-03-04T17:50:44.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.617+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=IX25TACJIK17 tag=MKSU75JRVHAV type=write size=2802020 2024-03-04T17:50:44.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.621+0000 7fda08af98c0 1 < canceled operation key=RXVSQVVNXVF0 tag=VW0VRBPJ3RSS type=del size=1995806 2024-03-04T17:50:44.624 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.625+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=CHZC9DLKE8R9 type=del size=190614 2024-03-04T17:50:44.628 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.629+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~CR0WTQMN43A9.0 size=1048576 2024-03-04T17:50:44.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.633+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~CR0WTQMN43A9.1 size=1048576 2024-03-04T17:50:44.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.637+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~CR0WTQMN43A9.2 size=1012555 2024-03-04T17:50:44.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.641+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=CR0WTQMN43A9 tag=MEGKOXUPXIRW type=write size=4158283 2024-03-04T17:50:44.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.645+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=IR0ZOC97HCC6 type=write size=365355 2024-03-04T17:50:44.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.653+0000 7fda08af98c0 1 < suggested update operation key=Z2JQBN84QQIB tag=FQEM7RROJDP4 type=write size=88212 2024-03-04T17:50:44.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.661+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~WQC0MCPC7APT.0 size=1048576 2024-03-04T17:50:44.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.665+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~WQC0MCPC7APT.1 size=1048576 2024-03-04T17:50:44.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.665+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~WQC0MCPC7APT.2 size=112398 2024-03-04T17:50:44.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.665+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=WQC0MCPC7APT tag=OK0GMSIUPOO9 type=write size=3258126 2024-03-04T17:50:44.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.673+0000 7fda08af98c0 1 < suggested update operation key=C1GK00FM2ZYA tag=4DEHNI1721Z4 type=del size=2153571 2024-03-04T17:50:44.677 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.677+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=P1JRAMDM7YKZ type=del size=3479280 2024-03-04T17:50:44.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.681+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~83K2SU8DJQ6C.0 size=1048576 2024-03-04T17:50:44.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.681+0000 7fda08af98c0 1 < canceled part key=_multipart_Z3J7W9V8HQKH.2~83K2SU8DJQ6C.1 size=218873 2024-03-04T17:50:44.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.689+0000 7fda08af98c0 1 < canceled multipart upload key=Z3J7W9V8HQKH upload=83K2SU8DJQ6C tag=BZTF4JPIBKAZ type=write size=2316025 2024-03-04T17:50:44.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.697+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=FG56DTXE2SWQ type=del size=3595632 2024-03-04T17:50:44.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.701+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~F9N3OSUL8K0Y.0 size=599007 2024-03-04T17:50:44.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.709+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=F9N3OSUL8K0Y tag=91DW58EDUM9W type=write size=1647583 2024-03-04T17:50:44.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.713+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=T75R2XCZQYAX type=write size=494217 2024-03-04T17:50:44.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.717+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=RF1Z2FXQ2CMD type=del 2024-03-04T17:50:44.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.721+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=2VCP4KWV9RHW type=del size=2309411 2024-03-04T17:50:44.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.725+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~CU4XX47LPESU.0 size=300579 2024-03-04T17:50:44.728 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.729+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=CU4XX47LPESU tag=QILQW0K2ZLTZ type=write size=1349155 2024-03-04T17:50:44.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.733+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=G2RQSJ3R5KMU type=write size=756343 2024-03-04T17:50:44.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.737+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~TX03A09TYO59.0 size=895755 2024-03-04T17:50:44.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.741+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=TX03A09TYO59 tag=VRU1QHLK5MZX type=write size=1944331 2024-03-04T17:50:44.745 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.745+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=3PIVZ3BEKTNC type=del size=1775166 2024-03-04T17:50:44.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.749+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~HO1NYNH1P20V.0 size=1048576 2024-03-04T17:50:44.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.753+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~HO1NYNH1P20V.1 size=4648 2024-03-04T17:50:44.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.761+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=HO1NYNH1P20V tag=FTKCCG4EY6YU type=write size=2101800 2024-03-04T17:50:44.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.765+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~4HSWQD0Y6OQW.0 size=1048576 2024-03-04T17:50:44.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.769+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~4HSWQD0Y6OQW.1 size=1048576 2024-03-04T17:50:44.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.773+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~4HSWQD0Y6OQW.2 size=784228 2024-03-04T17:50:44.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.773+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=4HSWQD0Y6OQW tag=PI0BCFOI1H1Q type=write size=3929956 2024-03-04T17:50:44.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.777+0000 7fda08af98c0 -1 < failed to complete operation key=Z2JQBN84QQIB tag=R3DJMFASKP8S type=write size=674122: (22) Invalid argument 2024-03-04T17:50:44.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.785+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~PJ8181G46013.0 size=117965 2024-03-04T17:50:44.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.785+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=PJ8181G46013 tag=4ZVCUBUVHEY8 type=write size=1166541 2024-03-04T17:50:44.786 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.789+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=06GB5Q1KAQCW type=del 2024-03-04T17:50:44.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.793+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=6MQGYN7JK6EJ type=del size=2465577 2024-03-04T17:50:44.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.797+0000 7fda08af98c0 1 < completed write operation key=ITIVF3UAT9NK tag=S46S6627FVM7 type=write size=211458 2024-03-04T17:50:44.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.805+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~IFT3O71W50MJ.0 size=196646 2024-03-04T17:50:44.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.805+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=IFT3O71W50MJ tag=ND3EYPG33HVP type=write size=1245222 2024-03-04T17:50:44.806 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.809+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~7RT4MYJI7TAX.0 size=1048576 2024-03-04T17:50:44.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.809+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~7RT4MYJI7TAX.1 size=1048576 2024-03-04T17:50:44.812 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.813+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~7RT4MYJI7TAX.2 size=218963 2024-03-04T17:50:44.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.821+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=7RT4MYJI7TAX tag=VLUIWDZ8KZ5L type=write size=3364691 2024-03-04T17:50:44.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.829+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~F5BW1DPSCN90.0 size=1048576 2024-03-04T17:50:44.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.829+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~F5BW1DPSCN90.1 size=487883 2024-03-04T17:50:44.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.829+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=F5BW1DPSCN90 tag=AFQVJ3USDY65 type=write size=2585035 2024-03-04T17:50:44.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.833+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~HVB8AOUMVV83.0 size=1048576 2024-03-04T17:50:44.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.837+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~HVB8AOUMVV83.1 size=655008 2024-03-04T17:50:44.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.845+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=HVB8AOUMVV83 tag=4HWAXZZNLEN4 type=write size=2752160 2024-03-04T17:50:44.848 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.849+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=X3C39ZFABKZ9 type=del size=2950056 2024-03-04T17:50:44.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.849+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~IX25TACJIK17.0 size=1048576 2024-03-04T17:50:44.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.853+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~IX25TACJIK17.1 size=704868 2024-03-04T17:50:44.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.861+0000 7fda08af98c0 1 < canceled multipart upload key=JA4JK99I0942 upload=IX25TACJIK17 tag=MKSU75JRVHAV type=write size=2802020 2024-03-04T17:50:44.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.865+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~EP0K52ENK5AH.0 size=1048576 2024-03-04T17:50:44.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.869+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~EP0K52ENK5AH.1 size=341829 2024-03-04T17:50:44.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.869+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=EP0K52ENK5AH tag=XTI3NKYJVB0J type=write size=2438981 2024-03-04T17:50:44.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.877+0000 7fda08af98c0 1 < suggested update operation key=0IGSMFORCSL9 tag=CHZC9DLKE8R9 type=del size=190614 2024-03-04T17:50:44.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.881+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=O5HP2VTSEIEX type=del size=1616624 2024-03-04T17:50:44.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.885+0000 7fda08af98c0 1 < completed write operation key=HCKJZUMAK0Z0 tag=IR0ZOC97HCC6 type=write size=365355 2024-03-04T17:50:44.886 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.889+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=8SIETVV7Y2IQ type=del size=1866974 2024-03-04T17:50:44.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.893+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~WQC0MCPC7APT.0 size=1048576 2024-03-04T17:50:44.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.893+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~WQC0MCPC7APT.1 size=1048576 2024-03-04T17:50:44.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.897+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~WQC0MCPC7APT.2 size=112398 2024-03-04T17:50:44.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.905+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=WQC0MCPC7APT tag=OK0GMSIUPOO9 type=write size=3258126 2024-03-04T17:50:44.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.909+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~BWC53EHIBOXY.0 size=1048576 2024-03-04T17:50:44.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.913+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~BWC53EHIBOXY.1 size=798117 2024-03-04T17:50:44.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.913+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=BWC53EHIBOXY tag=U46NPFUSXPCG type=write size=2895269 2024-03-04T17:50:44.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.917+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=P1JRAMDM7YKZ type=del 2024-03-04T17:50:44.919 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.921+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=T0CRSRMOV9TJ type=del size=2138160 2024-03-04T17:50:44.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.925+0000 7fda08af98c0 1 < canceled operation key=L0FVKWD7EQRW tag=FG56DTXE2SWQ type=del size=3595632 2024-03-04T17:50:44.927 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.929+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=2QQ7SMIQIQLG type=write size=894241 2024-03-04T17:50:44.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.933+0000 7fda08af98c0 1 < completed write operation key=M5BMX7B7RZYL tag=T75R2XCZQYAX type=write size=494217 2024-03-04T17:50:44.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.941+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~AX55PQ5A3DAO.0 size=1048576 2024-03-04T17:50:44.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.941+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~AX55PQ5A3DAO.1 size=1048576 2024-03-04T17:50:44.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.945+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~AX55PQ5A3DAO.2 size=400408 2024-03-04T17:50:44.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.945+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=AX55PQ5A3DAO tag=DSKR9DTML3LH type=write size=3546136 2024-03-04T17:50:44.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.949+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=2VCP4KWV9RHW type=del 2024-03-04T17:50:44.949 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.949+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=HSE6YRPGRTJT type=del size=1206590 2024-03-04T17:50:44.953 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.953+0000 7fda08af98c0 1 < completed write operation key=Z2JQBN84QQIB tag=G2RQSJ3R5KMU type=write size=756343 2024-03-04T17:50:44.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.957+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=BV25FKYKJDNG type=del size=2795893 2024-03-04T17:50:44.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.961+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=3PIVZ3BEKTNC type=del 2024-03-04T17:50:44.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.969+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~YGPR4T3MM07M.0 size=694674 2024-03-04T17:50:44.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.969+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=YGPR4T3MM07M tag=LFK8QNN25XEL type=write size=1743250 2024-03-04T17:50:44.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.973+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~4HSWQD0Y6OQW.0 size=1048576 2024-03-04T17:50:44.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.977+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~4HSWQD0Y6OQW.1 size=1048576 2024-03-04T17:50:44.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.977+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~4HSWQD0Y6OQW.2 size=784228 2024-03-04T17:50:44.982 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.985+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=4HSWQD0Y6OQW tag=PI0BCFOI1H1Q type=write size=3929956 2024-03-04T17:50:44.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.989+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~AR1GK2CWPJM5.0 size=482784 2024-03-04T17:50:44.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.989+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=AR1GK2CWPJM5 tag=EZ7VCNOGVAMT type=write size=1531360 2024-03-04T17:50:44.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:44.993+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~PJ8181G46013.0 size=117965 2024-03-04T17:50:44.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.001+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=PJ8181G46013 tag=4ZVCUBUVHEY8 type=write size=1166541 2024-03-04T17:50:45.003 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.005+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=KMWO45YBPTXQ type=write size=949031 2024-03-04T17:50:45.007 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.009+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=6MQGYN7JK6EJ type=del 2024-03-04T17:50:45.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.013+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=F14BT9X48IJR type=write size=805988 2024-03-04T17:50:45.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.017+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~IFT3O71W50MJ.0 size=196646 2024-03-04T17:50:45.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.021+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=IFT3O71W50MJ tag=ND3EYPG33HVP type=write size=1245222 2024-03-04T17:50:45.028 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.029+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~NHRMBRQ69D1R.0 size=1048576 2024-03-04T17:50:45.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.033+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~NHRMBRQ69D1R.1 size=1048576 2024-03-04T17:50:45.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.037+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~NHRMBRQ69D1R.2 size=525047 2024-03-04T17:50:45.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.037+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=NHRMBRQ69D1R tag=BH4SPWN1LHW3 type=write size=3670775 2024-03-04T17:50:45.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.041+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~F5BW1DPSCN90.0 size=1048576 2024-03-04T17:50:45.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.045+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~F5BW1DPSCN90.1 size=487883 2024-03-04T17:50:45.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.053+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=F5BW1DPSCN90 tag=AFQVJ3USDY65 type=write size=2585035 2024-03-04T17:50:45.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.057+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~AEM9UOEJ08QD.0 size=1048576 2024-03-04T17:50:45.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.061+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~AEM9UOEJ08QD.1 size=604571 2024-03-04T17:50:45.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.061+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=AEM9UOEJ08QD tag=BPTYNMMD5Z7G type=write size=2701723 2024-03-04T17:50:45.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.065+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=X3C39ZFABKZ9 type=del 2024-03-04T17:50:45.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.069+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~48IQKFEEX1DS.0 size=1048576 2024-03-04T17:50:45.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.073+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~48IQKFEEX1DS.1 size=194518 2024-03-04T17:50:45.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.073+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=48IQKFEEX1DS tag=VV43ZDJVFYHR type=write size=2291670 2024-03-04T17:50:45.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.073+0000 7fda08af98c0 1 < canceled part key=_multipart_HCA7QLGM2O4O.2~EP0K52ENK5AH.0 size=1048576 2024-03-04T17:50:45.076 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.077+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~EP0K52ENK5AH.1 size=341829 2024-03-04T17:50:45.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.085+0000 7fda08af98c0 1 < canceled multipart upload key=HCA7QLGM2O4O upload=EP0K52ENK5AH tag=XTI3NKYJVB0J type=write size=2438981 2024-03-04T17:50:45.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.089+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=BJXJNA7STSG0 type=del size=3655303 2024-03-04T17:50:45.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.093+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=O5HP2VTSEIEX type=del 2024-03-04T17:50:45.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.097+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=G8ERKTWY61DC type=del size=1105001 2024-03-04T17:50:45.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.105+0000 7fda08af98c0 1 < suggested remove operation key=DJCCQ6S0E7UG tag=8SIETVV7Y2IQ type=del 2024-03-04T17:50:45.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.109+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=1J6J5L1LETXR type=del size=2032342 2024-03-04T17:50:45.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.113+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~BWC53EHIBOXY.0 size=1048576 2024-03-04T17:50:45.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.113+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~BWC53EHIBOXY.1 size=798117 2024-03-04T17:50:45.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.121+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=BWC53EHIBOXY tag=U46NPFUSXPCG type=write size=2895269 2024-03-04T17:50:45.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.125+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=DRAK3NUV780E type=write size=186878 2024-03-04T17:50:45.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.129+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=T0CRSRMOV9TJ type=del 2024-03-04T17:50:45.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.133+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~P5CPRJEVEBCU.0 size=1048576 2024-03-04T17:50:45.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.137+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~P5CPRJEVEBCU.1 size=649953 2024-03-04T17:50:45.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.137+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=P5CPRJEVEBCU tag=XIO7M0RC19JQ type=write size=2747105 2024-03-04T17:50:45.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.141+0000 7fda08af98c0 1 < completed write operation key=TOFWDFH0HL3Q tag=2QQ7SMIQIQLG type=write size=894241 2024-03-04T17:50:45.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.145+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=E0TVN0SKRET5 type=del size=1610092 2024-03-04T17:50:45.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.149+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~AX55PQ5A3DAO.0 size=1048576 2024-03-04T17:50:45.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.153+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~AX55PQ5A3DAO.1 size=1048576 2024-03-04T17:50:45.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.153+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~AX55PQ5A3DAO.2 size=400408 2024-03-04T17:50:45.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.161+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=AX55PQ5A3DAO tag=DSKR9DTML3LH type=write size=3546136 2024-03-04T17:50:45.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.165+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=9UQCE93LOKYQ type=del size=3944468 2024-03-04T17:50:45.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.169+0000 7fda08af98c0 1 < canceled operation key=XG5I3SDRE5JM tag=HSE6YRPGRTJT type=del size=1206590 2024-03-04T17:50:45.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.177+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~3FN8MEF625U1.0 size=1048576 2024-03-04T17:50:45.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.181+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~3FN8MEF625U1.1 size=244004 2024-03-04T17:50:45.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.181+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=3FN8MEF625U1 tag=V1XNU6LLLGSZ type=write size=2341156 2024-03-04T17:50:45.181 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.181+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=BV25FKYKJDNG type=del 2024-03-04T17:50:45.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.189+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~S50GIGH2NOX1.0 size=1048576 2024-03-04T17:50:45.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.193+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~S50GIGH2NOX1.1 size=93176 2024-03-04T17:50:45.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.193+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=S50GIGH2NOX1 tag=BUDVUHLR5UJ1 type=write size=2190328 2024-03-04T17:50:45.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.197+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~YGPR4T3MM07M.0 size=694674 2024-03-04T17:50:45.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.201+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=YGPR4T3MM07M tag=LFK8QNN25XEL type=write size=1743250 2024-03-04T17:50:45.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.205+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=ZNOA082CY1WP type=del size=225474 2024-03-04T17:50:45.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.209+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~AR1GK2CWPJM5.0 size=482784 2024-03-04T17:50:45.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.217+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=AR1GK2CWPJM5 tag=EZ7VCNOGVAMT type=write size=1531360 2024-03-04T17:50:45.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.221+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=XQW9BP85ANPN type=del size=1804957 2024-03-04T17:50:45.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.225+0000 7fda08af98c0 1 < completed write operation key=C4B45GL40IMZ tag=KMWO45YBPTXQ type=write size=949031 2024-03-04T17:50:45.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.233+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~ZWA1DX1TPH3N.0 size=529257 2024-03-04T17:50:45.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.233+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=ZWA1DX1TPH3N tag=OMGCS3Z4WUU6 type=write size=1577833 2024-03-04T17:50:45.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.241+0000 7fda08af98c0 1 < suggested update operation key=09KBCA2XURTF tag=F14BT9X48IJR type=write size=805988 2024-03-04T17:50:45.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.245+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=1NTHCJWHPW1O type=del size=3488585 2024-03-04T17:50:45.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.249+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~NHRMBRQ69D1R.0 size=1048576 2024-03-04T17:50:45.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.253+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~NHRMBRQ69D1R.1 size=1048576 2024-03-04T17:50:45.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.257+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~NHRMBRQ69D1R.2 size=525047 2024-03-04T17:50:45.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.261+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=NHRMBRQ69D1R tag=BH4SPWN1LHW3 type=write size=3670775 2024-03-04T17:50:45.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.265+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=ITY2KUKVAB70 type=del size=889641 2024-03-04T17:50:45.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.269+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~AEM9UOEJ08QD.0 size=1048576 2024-03-04T17:50:45.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.273+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~AEM9UOEJ08QD.1 size=604571 2024-03-04T17:50:45.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.277+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=AEM9UOEJ08QD tag=BPTYNMMD5Z7G type=write size=2701723 2024-03-04T17:50:45.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.281+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=CIRI7VD2VC0Z type=write size=450596 2024-03-04T17:50:45.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.281+0000 7fda08af98c0 1 < canceled part key=_multipart_G0BHMSCO0XLF.2~48IQKFEEX1DS.0 size=1048576 2024-03-04T17:50:45.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.285+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~48IQKFEEX1DS.1 size=194518 2024-03-04T17:50:45.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.293+0000 7fda08af98c0 1 < canceled multipart upload key=G0BHMSCO0XLF upload=48IQKFEEX1DS tag=VV43ZDJVFYHR type=write size=2291670 2024-03-04T17:50:45.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.297+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~8KDJENM9NFXM.0 size=1048576 2024-03-04T17:50:45.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.301+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~8KDJENM9NFXM.1 size=1048576 2024-03-04T17:50:45.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.305+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~8KDJENM9NFXM.2 size=675102 2024-03-04T17:50:45.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.305+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=8KDJENM9NFXM tag=J5UZQY9C62US type=write size=3820830 2024-03-04T17:50:45.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.309+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=BJXJNA7STSG0 type=del 2024-03-04T17:50:45.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.317+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~O5VRY47YU5TR.0 size=340253 2024-03-04T17:50:45.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.317+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=O5VRY47YU5TR tag=95KE879NGKBV type=write size=1388829 2024-03-04T17:50:45.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.317+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=G8ERKTWY61DC type=del 2024-03-04T17:50:45.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.325+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~W3FVCZ0EW9PW.0 size=1048576 2024-03-04T17:50:45.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.329+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~W3FVCZ0EW9PW.1 size=1048576 2024-03-04T17:50:45.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.333+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~W3FVCZ0EW9PW.2 size=905615 2024-03-04T17:50:45.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.333+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=W3FVCZ0EW9PW tag=5ED8ER7HINU4 type=write size=4051343 2024-03-04T17:50:45.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.337+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=1J6J5L1LETXR type=del 2024-03-04T17:50:45.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.341+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~345AS7A8VMT2.0 size=1048576 2024-03-04T17:50:45.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.345+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~345AS7A8VMT2.1 size=87355 2024-03-04T17:50:45.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.345+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=345AS7A8VMT2 tag=9UU82E2GV1BH type=write size=2184507 2024-03-04T17:50:45.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.349+0000 7fda08af98c0 1 < completed write operation key=750NSVFBS03A tag=DRAK3NUV780E type=write size=186878 2024-03-04T17:50:45.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.353+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=URUVUZ37DVUS type=del size=4178756 2024-03-04T17:50:45.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.357+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~P5CPRJEVEBCU.0 size=1048576 2024-03-04T17:50:45.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.361+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~P5CPRJEVEBCU.1 size=649953 2024-03-04T17:50:45.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.365+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=P5CPRJEVEBCU tag=XIO7M0RC19JQ type=write size=2747105 2024-03-04T17:50:45.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.369+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=7JZNX2VLAW61 type=del size=2937794 2024-03-04T17:50:45.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.373+0000 7fda08af98c0 1 < canceled operation key=RXVSQVVNXVF0 tag=E0TVN0SKRET5 type=del size=1610092 2024-03-04T17:50:45.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.377+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=A1O36LG87PK4 type=del size=585204 2024-03-04T17:50:45.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.381+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=9UQCE93LOKYQ type=del 2024-03-04T17:50:45.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.389+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~Y56PPBIMALHW.0 size=261184 2024-03-04T17:50:45.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.389+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=Y56PPBIMALHW tag=7YXR3C4J81NF type=write size=1309760 2024-03-04T17:50:45.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.389+0000 7fda08af98c0 1 < canceled part key=_multipart_C4B45GL40IMZ.2~3FN8MEF625U1.0 size=1048576 2024-03-04T17:50:45.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.389+0000 7fda08af98c0 1 < canceled part key=_multipart_C4B45GL40IMZ.2~3FN8MEF625U1.1 size=244004 2024-03-04T17:50:45.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.401+0000 7fda08af98c0 1 < canceled multipart upload key=C4B45GL40IMZ upload=3FN8MEF625U1 tag=V1XNU6LLLGSZ type=write size=2341156 2024-03-04T17:50:45.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.405+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=89V1ELGTO9PZ type=del size=1511414 2024-03-04T17:50:45.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.409+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~S50GIGH2NOX1.0 size=1048576 2024-03-04T17:50:45.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.409+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~S50GIGH2NOX1.1 size=93176 2024-03-04T17:50:45.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.417+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=S50GIGH2NOX1 tag=BUDVUHLR5UJ1 type=write size=2190328 2024-03-04T17:50:45.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.425+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~982TD5Y511V8.0 size=1048576 2024-03-04T17:50:45.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.429+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~982TD5Y511V8.1 size=443761 2024-03-04T17:50:45.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.429+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=982TD5Y511V8 tag=ZILLB61GRROH type=write size=2540913 2024-03-04T17:50:45.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.433+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=ZNOA082CY1WP type=del 2024-03-04T17:50:45.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.437+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=YIFU2EPWLFS9 type=del size=771288 2024-03-04T17:50:45.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.437+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=XQW9BP85ANPN type=del 2024-03-04T17:50:45.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.445+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~CPXROA546FUW.0 size=1048576 2024-03-04T17:50:45.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.449+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~CPXROA546FUW.1 size=1048576 2024-03-04T17:50:45.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.449+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~CPXROA546FUW.2 size=449051 2024-03-04T17:50:45.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.449+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=CPXROA546FUW tag=482ROVYUUEW7 type=write size=3594779 2024-03-04T17:50:45.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.453+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~ZWA1DX1TPH3N.0 size=529257 2024-03-04T17:50:45.458 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.461+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=ZWA1DX1TPH3N tag=OMGCS3Z4WUU6 type=write size=1577833 2024-03-04T17:50:45.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.465+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~PFDAPN56RUGJ.0 size=1048576 2024-03-04T17:50:45.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.469+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~PFDAPN56RUGJ.1 size=802694 2024-03-04T17:50:45.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.469+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=PFDAPN56RUGJ tag=MRN1JO2VOHVE type=write size=2899846 2024-03-04T17:50:45.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.477+0000 7fda08af98c0 1 < suggested remove operation key=2PA8JDZUOLO5 tag=1NTHCJWHPW1O type=del 2024-03-04T17:50:45.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.477+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=C8MABNFN1VUI type=del size=1743967 2024-03-04T17:50:45.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.485+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=ITY2KUKVAB70 type=del 2024-03-04T17:50:45.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.489+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~DQ505WVETMAE.0 size=730543 2024-03-04T17:50:45.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.489+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=DQ505WVETMAE tag=GHP8WIRPOYK6 type=write size=1779119 2024-03-04T17:50:45.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.497+0000 7fda08af98c0 1 < suggested update operation key=JA4JK99I0942 tag=CIRI7VD2VC0Z type=write size=450596 2024-03-04T17:50:45.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.505+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~OHW47429PZCK.0 size=1048576 2024-03-04T17:50:45.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.505+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~OHW47429PZCK.1 size=168862 2024-03-04T17:50:45.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.505+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=OHW47429PZCK tag=ML7XRF79M0B5 type=write size=2266014 2024-03-04T17:50:45.509 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.509+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~8KDJENM9NFXM.0 size=1048576 2024-03-04T17:50:45.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.513+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~8KDJENM9NFXM.1 size=1048576 2024-03-04T17:50:45.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.517+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~8KDJENM9NFXM.2 size=675102 2024-03-04T17:50:45.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.521+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=8KDJENM9NFXM tag=J5UZQY9C62US type=write size=3820830 2024-03-04T17:50:45.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.525+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=9WQLU3WL120V type=del size=2767635 2024-03-04T17:50:45.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.529+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~O5VRY47YU5TR.0 size=340253 2024-03-04T17:50:45.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.533+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=O5VRY47YU5TR tag=95KE879NGKBV type=write size=1388829 2024-03-04T17:50:45.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.541+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~TU3VOV1E0VY2.0 size=1048576 2024-03-04T17:50:45.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.545+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~TU3VOV1E0VY2.1 size=1048576 2024-03-04T17:50:45.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.549+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~TU3VOV1E0VY2.2 size=516829 2024-03-04T17:50:45.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.549+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=TU3VOV1E0VY2 tag=Z9KZ533IO88B type=write size=3662557 2024-03-04T17:50:45.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.549+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~W3FVCZ0EW9PW.0 size=1048576 2024-03-04T17:50:45.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.549+0000 7fda08af98c0 1 < canceled part key=_multipart_C4B45GL40IMZ.2~W3FVCZ0EW9PW.1 size=1048576 2024-03-04T17:50:45.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.553+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~W3FVCZ0EW9PW.2 size=905615 2024-03-04T17:50:45.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.557+0000 7fda08af98c0 1 < canceled multipart upload key=C4B45GL40IMZ upload=W3FVCZ0EW9PW tag=5ED8ER7HINU4 type=write size=4051343 2024-03-04T17:50:45.563 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.565+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~KW0TGJD84N3E.0 size=1048576 2024-03-04T17:50:45.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.569+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~KW0TGJD84N3E.1 size=63510 2024-03-04T17:50:45.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.569+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=KW0TGJD84N3E tag=7SGT9ELCMMN8 type=write size=2160662 2024-03-04T17:50:45.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.569+0000 7fda08af98c0 1 < canceled part key=_multipart_1OZKPHMPV3OQ.2~345AS7A8VMT2.0 size=1048576 2024-03-04T17:50:45.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.573+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~345AS7A8VMT2.1 size=87355 2024-03-04T17:50:45.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.577+0000 7fda08af98c0 1 < canceled multipart upload key=1OZKPHMPV3OQ upload=345AS7A8VMT2 tag=9UU82E2GV1BH type=write size=2184507 2024-03-04T17:50:45.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.581+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=I0DGJM3RJ52Z type=del size=1004214 2024-03-04T17:50:45.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.593+0000 7fda08af98c0 1 < suggested update operation key=0IGSMFORCSL9 tag=URUVUZ37DVUS type=del size=4178756 2024-03-04T17:50:45.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.597+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=VY7UI99CXEUB type=write size=429623 2024-03-04T17:50:45.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.605+0000 7fda08af98c0 1 < suggested update operation key=VJVORUG95FLC tag=7JZNX2VLAW61 type=del size=2937794 2024-03-04T17:50:45.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.609+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=JWGV0YXRZ7IN type=del size=3684023 2024-03-04T17:50:45.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.613+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=A1O36LG87PK4 type=del 2024-03-04T17:50:45.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.621+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~BBHOB5K04UYP.0 size=1048576 2024-03-04T17:50:45.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.625+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~BBHOB5K04UYP.1 size=1048576 2024-03-04T17:50:45.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.629+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~BBHOB5K04UYP.2 size=823900 2024-03-04T17:50:45.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.629+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=BBHOB5K04UYP tag=8AXAR0J459OD type=write size=3969628 2024-03-04T17:50:45.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.633+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~Y56PPBIMALHW.0 size=261184 2024-03-04T17:50:45.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.637+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=Y56PPBIMALHW tag=7YXR3C4J81NF type=write size=1309760 2024-03-04T17:50:45.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.641+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=XXPC5GWBEIBW type=del size=3924904 2024-03-04T17:50:45.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.645+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=89V1ELGTO9PZ type=del 2024-03-04T17:50:45.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.653+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~UNYEZOM4G9UX.0 size=1048576 2024-03-04T17:50:45.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.657+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~UNYEZOM4G9UX.1 size=1048576 2024-03-04T17:50:45.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.661+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~UNYEZOM4G9UX.2 size=485727 2024-03-04T17:50:45.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.661+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=UNYEZOM4G9UX tag=F9OHM2TCBKDC type=write size=3631455 2024-03-04T17:50:45.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.665+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~982TD5Y511V8.0 size=1048576 2024-03-04T17:50:45.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.669+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~982TD5Y511V8.1 size=443761 2024-03-04T17:50:45.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.677+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=982TD5Y511V8 tag=ZILLB61GRROH type=write size=2540913 2024-03-04T17:50:45.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.685+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8TKIJ51S4A80.0 size=1048576 2024-03-04T17:50:45.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.685+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8TKIJ51S4A80.1 size=413876 2024-03-04T17:50:45.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.685+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=8TKIJ51S4A80 tag=5517HM7061Z8 type=write size=2511028 2024-03-04T17:50:45.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.693+0000 7fda08af98c0 1 < suggested remove operation key=IJSYAVR5SWPS tag=YIFU2EPWLFS9 type=del 2024-03-04T17:50:45.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.701+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~0DIQ87GFLX18.0 size=1048576 2024-03-04T17:50:45.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.705+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~0DIQ87GFLX18.1 size=920560 2024-03-04T17:50:45.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.705+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=0DIQ87GFLX18 tag=VC45TCKB4884 type=write size=3017712 2024-03-04T17:50:45.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.709+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~CPXROA546FUW.0 size=1048576 2024-03-04T17:50:45.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.713+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~CPXROA546FUW.1 size=1048576 2024-03-04T17:50:45.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.717+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~CPXROA546FUW.2 size=449051 2024-03-04T17:50:45.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.721+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=CPXROA546FUW tag=482ROVYUUEW7 type=write size=3594779 2024-03-04T17:50:45.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.729+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=42KSPYN3GTRX type=write size=515315 2024-03-04T17:50:45.729 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.733+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~PFDAPN56RUGJ.0 size=1048576 2024-03-04T17:50:45.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.733+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~PFDAPN56RUGJ.1 size=802694 2024-03-04T17:50:45.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.741+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=PFDAPN56RUGJ tag=MRN1JO2VOHVE type=write size=2899846 2024-03-04T17:50:45.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.753+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~ERP9G83VY8Y2.0 size=927383 2024-03-04T17:50:45.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.753+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=ERP9G83VY8Y2 tag=HA0K0H8CA6VU type=write size=1975959 2024-03-04T17:50:45.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.757+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=C8MABNFN1VUI type=del 2024-03-04T17:50:45.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.761+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=8AIPQRDGA0YG type=del size=2937202 2024-03-04T17:50:45.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.765+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~DQ505WVETMAE.0 size=730543 2024-03-04T17:50:45.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.773+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=DQ505WVETMAE tag=GHP8WIRPOYK6 type=write size=1779119 2024-03-04T17:50:45.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.777+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=4MGS7Q7ZJ74Q type=del size=353526 2024-03-04T17:50:45.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.781+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~OHW47429PZCK.0 size=1048576 2024-03-04T17:50:45.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.785+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~OHW47429PZCK.1 size=168862 2024-03-04T17:50:45.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.789+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=OHW47429PZCK tag=ML7XRF79M0B5 type=write size=2266014 2024-03-04T17:50:45.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.793+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=CNVRKMTF8B8K type=del size=4093583 2024-03-04T17:50:45.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.797+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=9WQLU3WL120V type=del 2024-03-04T17:50:45.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.801+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=AYQIJWC31BMP type=del size=3898934 2024-03-04T17:50:45.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.805+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~TU3VOV1E0VY2.0 size=1048576 2024-03-04T17:50:45.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.809+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~TU3VOV1E0VY2.1 size=1048576 2024-03-04T17:50:45.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.813+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~TU3VOV1E0VY2.2 size=516829 2024-03-04T17:50:45.816 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.817+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=TU3VOV1E0VY2 tag=Z9KZ533IO88B type=write size=3662557 2024-03-04T17:50:45.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.825+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~JSHSWGDPP9WB.0 size=1048576 2024-03-04T17:50:45.827 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.829+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~JSHSWGDPP9WB.1 size=1048576 2024-03-04T17:50:45.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.833+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~JSHSWGDPP9WB.2 size=124635 2024-03-04T17:50:45.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.833+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=JSHSWGDPP9WB tag=RTRPZ5URALWQ type=write size=3270363 2024-03-04T17:50:45.830 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.833+0000 7fda08af98c0 1 < canceled part key=_multipart_8YL7C9PI58TK.2~KW0TGJD84N3E.0 size=1048576 2024-03-04T17:50:45.830 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.833+0000 7fda08af98c0 1 < canceled part key=_multipart_8YL7C9PI58TK.2~KW0TGJD84N3E.1 size=63510 2024-03-04T17:50:45.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.837+0000 7fda08af98c0 1 < canceled multipart upload key=8YL7C9PI58TK upload=KW0TGJD84N3E tag=7SGT9ELCMMN8 type=write size=2160662 2024-03-04T17:50:45.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.845+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~HLLLS04TA5DK.0 size=1048576 2024-03-04T17:50:45.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.849+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~HLLLS04TA5DK.1 size=1048576 2024-03-04T17:50:45.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.853+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~HLLLS04TA5DK.2 size=161170 2024-03-04T17:50:45.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.853+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=HLLLS04TA5DK tag=P6WARNZGDJIY type=write size=3306898 2024-03-04T17:50:45.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.857+0000 7fda08af98c0 1 < suggested update operation key=0IGSMFORCSL9 tag=I0DGJM3RJ52Z type=del size=1004214 2024-03-04T17:50:45.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.861+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=SCAPQKROPWFB type=del size=3644484 2024-03-04T17:50:45.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.869+0000 7fda08af98c0 1 < suggested remove operation key=ITIVF3UAT9NK tag=VY7UI99CXEUB type=write 2024-03-04T17:50:45.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.873+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=D0A61JGCLQKY type=write size=193937 2024-03-04T17:50:45.876 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.877+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=JWGV0YXRZ7IN type=del 2024-03-04T17:50:45.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.881+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=RUS5AC5JVKL5 type=del size=299025 2024-03-04T17:50:45.885 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.885+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~BBHOB5K04UYP.0 size=1048576 2024-03-04T17:50:45.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.889+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~BBHOB5K04UYP.1 size=1048576 2024-03-04T17:50:45.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.893+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~BBHOB5K04UYP.2 size=823900 2024-03-04T17:50:45.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.901+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=BBHOB5K04UYP tag=8AXAR0J459OD type=write size=3969628 2024-03-04T17:50:45.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.901+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=2TLSQI0FSQDQ type=write size=45443 2024-03-04T17:50:45.905 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.905+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=XXPC5GWBEIBW type=del 2024-03-04T17:50:45.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.909+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=0ZMPSCY153CZ type=del size=3792929 2024-03-04T17:50:45.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.917+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~UNYEZOM4G9UX.0 size=1048576 2024-03-04T17:50:45.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.917+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~UNYEZOM4G9UX.1 size=1048576 2024-03-04T17:50:45.919 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.921+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~UNYEZOM4G9UX.2 size=485727 2024-03-04T17:50:45.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.929+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=UNYEZOM4G9UX tag=F9OHM2TCBKDC type=write size=3631455 2024-03-04T17:50:45.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.933+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=J4ZL45D9EVSD type=del size=3883412 2024-03-04T17:50:45.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.937+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~8TKIJ51S4A80.0 size=1048576 2024-03-04T17:50:45.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.937+0000 7fda08af98c0 1 < canceled part key=_multipart_VJVORUG95FLC.2~8TKIJ51S4A80.1 size=413876 2024-03-04T17:50:45.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.941+0000 7fda08af98c0 1 < canceled multipart upload key=VJVORUG95FLC upload=8TKIJ51S4A80 tag=5517HM7061Z8 type=write size=2511028 2024-03-04T17:50:45.944 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.945+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=ZEFD9LM3HKIB type=del size=2032146 2024-03-04T17:50:45.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.949+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~0DIQ87GFLX18.0 size=1048576 2024-03-04T17:50:45.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.949+0000 7fda08af98c0 1 < canceled part key=_multipart_M5BMX7B7RZYL.2~0DIQ87GFLX18.1 size=920560 2024-03-04T17:50:45.955 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.957+0000 7fda08af98c0 1 < canceled multipart upload key=M5BMX7B7RZYL upload=0DIQ87GFLX18 tag=VC45TCKB4884 type=write size=3017712 2024-03-04T17:50:45.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.961+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=X7C45V9JPAKA type=del size=3877767 2024-03-04T17:50:45.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.969+0000 7fda08af98c0 1 < suggested remove operation key=VJVORUG95FLC tag=42KSPYN3GTRX type=write 2024-03-04T17:50:45.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.973+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~89TRT1BWN0L0.0 size=1048576 2024-03-04T17:50:45.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.977+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~89TRT1BWN0L0.1 size=465809 2024-03-04T17:50:45.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.977+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=89TRT1BWN0L0 tag=ILZNAYXAAE0R type=write size=2562961 2024-03-04T17:50:45.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.981+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~ERP9G83VY8Y2.0 size=927383 2024-03-04T17:50:45.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.989+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=ERP9G83VY8Y2 tag=HA0K0H8CA6VU type=write size=1975959 2024-03-04T17:50:45.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.993+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~J54PL41ISKHQ.0 size=1048576 2024-03-04T17:50:45.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.997+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~J54PL41ISKHQ.1 size=560833 2024-03-04T17:50:45.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:45.997+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=J54PL41ISKHQ tag=63LM2EJOEHOD type=write size=2657985 2024-03-04T17:50:45.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.001+0000 7fda08af98c0 1 < canceled operation key=2PA8JDZUOLO5 tag=8AIPQRDGA0YG type=del size=2937202 2024-03-04T17:50:46.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.009+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~FX7A8Z2QL3L8.0 size=17524 2024-03-04T17:50:46.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.009+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=FX7A8Z2QL3L8 tag=6EU7M0YN3EB1 type=write size=1066100 2024-03-04T17:50:46.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.009+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=4MGS7Q7ZJ74Q type=del 2024-03-04T17:50:46.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.017+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~1HF6GHZKI3JO.0 size=1048576 2024-03-04T17:50:46.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.021+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~1HF6GHZKI3JO.1 size=404174 2024-03-04T17:50:46.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.021+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=1HF6GHZKI3JO tag=WQVWJXFHZSKE type=write size=2501326 2024-03-04T17:50:46.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.025+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=CNVRKMTF8B8K type=del 2024-03-04T17:50:46.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.033+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~KTRB72D3ZA9J.0 size=195518 2024-03-04T17:50:46.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.033+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=KTRB72D3ZA9J tag=4HMZCTCVT14J type=write size=1244094 2024-03-04T17:50:46.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.037+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=AYQIJWC31BMP type=del 2024-03-04T17:50:46.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.041+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=YS1PT8XDTF8H type=del size=1954213 2024-03-04T17:50:46.042 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.045+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~JSHSWGDPP9WB.0 size=1048576 2024-03-04T17:50:46.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.049+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~JSHSWGDPP9WB.1 size=1048576 2024-03-04T17:50:46.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.049+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~JSHSWGDPP9WB.2 size=124635 2024-03-04T17:50:46.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.057+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=JSHSWGDPP9WB tag=RTRPZ5URALWQ type=write size=3270363 2024-03-04T17:50:46.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.061+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=KAUJVJV2Z05Z type=write size=50571 2024-03-04T17:50:46.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.065+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~HLLLS04TA5DK.0 size=1048576 2024-03-04T17:50:46.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.069+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~HLLLS04TA5DK.1 size=1048576 2024-03-04T17:50:46.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.073+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~HLLLS04TA5DK.2 size=161170 2024-03-04T17:50:46.077 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.077+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=HLLLS04TA5DK tag=P6WARNZGDJIY type=write size=3306898 2024-03-04T17:50:46.084 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.085+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~11U72DGEWV40.0 size=272076 2024-03-04T17:50:46.084 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.085+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=11U72DGEWV40 tag=ZXECIWZ1N2RP type=write size=1320652 2024-03-04T17:50:46.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.089+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=SCAPQKROPWFB type=del 2024-03-04T17:50:46.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.097+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~PTX0PWBZE6XJ.0 size=962192 2024-03-04T17:50:46.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.097+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=PTX0PWBZE6XJ tag=9DZATSMCE14L type=write size=2010768 2024-03-04T17:50:46.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.101+0000 7fda08af98c0 1 < completed write operation key=FNWA2N9U14HO tag=D0A61JGCLQKY type=write size=193937 2024-03-04T17:50:46.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.109+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~VUV35DCBWL8G.0 size=984299 2024-03-04T17:50:46.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.109+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=VUV35DCBWL8G tag=O1FRGBHDDMYQ type=write size=2032875 2024-03-04T17:50:46.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.113+0000 7fda08af98c0 -1 < failed to complete operation key=VJVORUG95FLC tag=RUS5AC5JVKL5 type=del size=299025: (22) Invalid argument 2024-03-04T17:50:46.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.121+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~ZHSBWXMQBTMA.0 size=1048576 2024-03-04T17:50:46.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.125+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~ZHSBWXMQBTMA.1 size=1048576 2024-03-04T17:50:46.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.125+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~ZHSBWXMQBTMA.2 size=454976 2024-03-04T17:50:46.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.125+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=ZHSBWXMQBTMA tag=F5RCNXC4XDZL type=write size=3600704 2024-03-04T17:50:46.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.133+0000 7fda08af98c0 1 < suggested remove operation key=C4B45GL40IMZ tag=2TLSQI0FSQDQ type=write 2024-03-04T17:50:46.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.137+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=UBZ3EKMMMG47 type=del size=515601 2024-03-04T17:50:46.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.141+0000 7fda08af98c0 1 < canceled operation key=TOFWDFH0HL3Q tag=0ZMPSCY153CZ type=del size=3792929 2024-03-04T17:50:46.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.149+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~3Q0ZR8N7C235.0 size=1048576 2024-03-04T17:50:46.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.153+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~3Q0ZR8N7C235.1 size=534024 2024-03-04T17:50:46.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.153+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=3Q0ZR8N7C235 tag=0WEJH62MRFT2 type=write size=2631176 2024-03-04T17:50:46.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.153+0000 7fda08af98c0 -1 < failed to complete operation key=C4B45GL40IMZ tag=J4ZL45D9EVSD type=del size=3883412: (22) Invalid argument 2024-03-04T17:50:46.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.157+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=R00FCSC5S4SW type=write size=119777 2024-03-04T17:50:46.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.161+0000 7fda08af98c0 1 < canceled operation key=09KBCA2XURTF tag=ZEFD9LM3HKIB type=del size=2032146 2024-03-04T17:50:46.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.169+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~CY9JT0G7VWIH.0 size=1023460 2024-03-04T17:50:46.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.169+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=CY9JT0G7VWIH tag=LUGW93ZF9EQ6 type=write size=2072036 2024-03-04T17:50:46.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.173+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=X7C45V9JPAKA type=del 2024-03-04T17:50:46.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.181+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~4ALBMH2YPD9B.0 size=226963 2024-03-04T17:50:46.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.181+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=4ALBMH2YPD9B tag=INWM6ZVD4ZNO type=write size=1275539 2024-03-04T17:50:46.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.185+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~89TRT1BWN0L0.0 size=1048576 2024-03-04T17:50:46.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.189+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~89TRT1BWN0L0.1 size=465809 2024-03-04T17:50:46.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.193+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=89TRT1BWN0L0 tag=ILZNAYXAAE0R type=write size=2562961 2024-03-04T17:50:46.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.197+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=4NG2RDH80TG2 type=del size=1318258 2024-03-04T17:50:46.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.201+0000 7fda08af98c0 1 < canceled part key=_multipart_FMPZ2548LBBF.2~J54PL41ISKHQ.0 size=1048576 2024-03-04T17:50:46.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.201+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~J54PL41ISKHQ.1 size=560833 2024-03-04T17:50:46.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.209+0000 7fda08af98c0 1 < canceled multipart upload key=FMPZ2548LBBF upload=J54PL41ISKHQ tag=63LM2EJOEHOD type=write size=2657985 2024-03-04T17:50:46.214 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.217+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~53CVU4P8462W.0 size=1048576 2024-03-04T17:50:46.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.221+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~53CVU4P8462W.1 size=522156 2024-03-04T17:50:46.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.221+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=53CVU4P8462W tag=TYB509JOYYQ4 type=write size=2619308 2024-03-04T17:50:46.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.225+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~FX7A8Z2QL3L8.0 size=17524 2024-03-04T17:50:46.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.229+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=FX7A8Z2QL3L8 tag=6EU7M0YN3EB1 type=write size=1066100 2024-03-04T17:50:46.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.237+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~WBLI3IJHRZX4.0 size=1045656 2024-03-04T17:50:46.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.237+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=WBLI3IJHRZX4 tag=RBZEZ32PMJYM type=write size=2094232 2024-03-04T17:50:46.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.245+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~1HF6GHZKI3JO.0 size=1048576 2024-03-04T17:50:46.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.245+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~1HF6GHZKI3JO.1 size=404174 2024-03-04T17:50:46.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.253+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=1HF6GHZKI3JO tag=WQVWJXFHZSKE type=write size=2501326 2024-03-04T17:50:46.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.261+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~Q307CCK2USKA.0 size=591539 2024-03-04T17:50:46.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.261+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=Q307CCK2USKA tag=MCBW61EWWY6S type=write size=1640115 2024-03-04T17:50:46.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.265+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~KTRB72D3ZA9J.0 size=195518 2024-03-04T17:50:46.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.273+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=KTRB72D3ZA9J tag=4HMZCTCVT14J type=write size=1244094 2024-03-04T17:50:46.277 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.277+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~D0UIXZLX4863.0 size=201795 2024-03-04T17:50:46.277 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.277+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=D0UIXZLX4863 tag=RQJPZFMUGB8H type=write size=1250371 2024-03-04T17:50:46.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.281+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=YS1PT8XDTF8H type=del 2024-03-04T17:50:46.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.289+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=52ON34M3L7YD type=del size=394843 2024-03-04T17:50:46.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.293+0000 7fda08af98c0 1 < canceled operation key=XG2XYBCMUNY5 tag=KAUJVJV2Z05Z type=write size=50571 2024-03-04T17:50:46.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.297+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=SFJJTE3G66JX type=del size=3669919 2024-03-04T17:50:46.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.297+0000 7fda08af98c0 1 < canceled part key=_multipart_XG5I3SDRE5JM.2~11U72DGEWV40.0 size=272076 2024-03-04T17:50:46.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.305+0000 7fda08af98c0 1 < canceled multipart upload key=XG5I3SDRE5JM upload=11U72DGEWV40 tag=ZXECIWZ1N2RP type=write size=1320652 2024-03-04T17:50:46.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.313+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~CDX8HQPEPJAN.0 size=1048576 2024-03-04T17:50:46.312 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.313+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~CDX8HQPEPJAN.1 size=994469 2024-03-04T17:50:46.312 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.313+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=CDX8HQPEPJAN tag=ARCY9NKD20TW type=write size=3091621 2024-03-04T17:50:46.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.317+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~PTX0PWBZE6XJ.0 size=962192 2024-03-04T17:50:46.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.325+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=PTX0PWBZE6XJ tag=9DZATSMCE14L type=write size=2010768 2024-03-04T17:50:46.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.329+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=4MM8YNBT4LS0 type=del size=1877573 2024-03-04T17:50:46.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.333+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~VUV35DCBWL8G.0 size=984299 2024-03-04T17:50:46.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.341+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=VUV35DCBWL8G tag=O1FRGBHDDMYQ type=write size=2032875 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.345+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=WSFW311LRXGI type=del size=1767455 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.349+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~ZHSBWXMQBTMA.0 size=1048576 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.349+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~ZHSBWXMQBTMA.1 size=1048576 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.353+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~ZHSBWXMQBTMA.2 size=454976 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.361+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=ZHSBWXMQBTMA tag=F5RCNXC4XDZL type=write size=3600704 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.365+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=N2KS2T0QUBJ0 type=del size=1802364 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.373+0000 7fda08af98c0 1 < suggested update operation key=TOFWDFH0HL3Q tag=UBZ3EKMMMG47 type=del size=515601 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.377+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=I7HNT3HHKDZK type=del size=545716 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.381+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~3Q0ZR8N7C235.0 size=1048576 2024-03-04T17:50:46.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.381+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~3Q0ZR8N7C235.1 size=534024 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.389+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=3Q0ZR8N7C235 tag=0WEJH62MRFT2 type=write size=2631176 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.397+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~3E84ZYDYS6V2.0 size=1048576 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.397+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~3E84ZYDYS6V2.1 size=1048576 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.401+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~3E84ZYDYS6V2.2 size=723794 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.401+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=3E84ZYDYS6V2 tag=SYVVPYQF9E8J type=write size=3869522 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.409+0000 7fda08af98c0 1 < suggested remove operation key=TOFWDFH0HL3Q tag=R00FCSC5S4SW type=write 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.417+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~JMX6GUTNATKA.0 size=1048576 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.417+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~JMX6GUTNATKA.1 size=61963 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.417+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=JMX6GUTNATKA tag=L79TGMPCO0WI type=write size=2159115 2024-03-04T17:50:46.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.421+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~CY9JT0G7VWIH.0 size=1023460 2024-03-04T17:50:46.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.429+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=CY9JT0G7VWIH tag=LUGW93ZF9EQ6 type=write size=2072036 2024-03-04T17:50:46.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.433+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~3E4AQ1S2BI4E.0 size=1048576 2024-03-04T17:50:46.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.437+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~3E4AQ1S2BI4E.1 size=614667 2024-03-04T17:50:46.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.437+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=3E4AQ1S2BI4E tag=ZIVGTC9SX2QP type=write size=2711819 2024-03-04T17:50:46.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.441+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~4ALBMH2YPD9B.0 size=226963 2024-03-04T17:50:46.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.449+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=4ALBMH2YPD9B tag=INWM6ZVD4ZNO type=write size=1275539 2024-03-04T17:50:46.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.453+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~6KZWYKOS5V7K.0 size=1048576 2024-03-04T17:50:46.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.457+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~6KZWYKOS5V7K.1 size=638680 2024-03-04T17:50:46.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.457+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=6KZWYKOS5V7K tag=SS5R0EWZFRS8 type=write size=2735832 2024-03-04T17:50:46.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.465+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=4NG2RDH80TG2 type=del 2024-03-04T17:50:46.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.473+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=KMGG7OQD1TYQ type=write size=114613 2024-03-04T17:50:46.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.473+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~53CVU4P8462W.0 size=1048576 2024-03-04T17:50:46.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.477+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~53CVU4P8462W.1 size=522156 2024-03-04T17:50:46.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.481+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=53CVU4P8462W tag=TYB509JOYYQ4 type=write size=2619308 2024-03-04T17:50:46.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.489+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~9G7OZV1HJ5G2.0 size=1048576 2024-03-04T17:50:46.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.493+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~9G7OZV1HJ5G2.1 size=1048576 2024-03-04T17:50:46.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.493+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~9G7OZV1HJ5G2.2 size=545897 2024-03-04T17:50:46.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.493+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=9G7OZV1HJ5G2 tag=P6ZW80OVJ9OJ type=write size=3691625 2024-03-04T17:50:46.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.501+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~WBLI3IJHRZX4.0 size=1045656 2024-03-04T17:50:46.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.505+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=WBLI3IJHRZX4 tag=RBZEZ32PMJYM type=write size=2094232 2024-03-04T17:50:46.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.513+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~7TQ55WEGQX1J.0 size=1048576 2024-03-04T17:50:46.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.513+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~7TQ55WEGQX1J.1 size=870545 2024-03-04T17:50:46.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.513+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=7TQ55WEGQX1J tag=0MDC20WNEET4 type=write size=2967697 2024-03-04T17:50:46.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.517+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~Q307CCK2USKA.0 size=591539 2024-03-04T17:50:46.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.525+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=Q307CCK2USKA tag=MCBW61EWWY6S type=write size=1640115 2024-03-04T17:50:46.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.529+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=4W06I87Q0QLW type=del size=3281998 2024-03-04T17:50:46.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.533+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~D0UIXZLX4863.0 size=201795 2024-03-04T17:50:46.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.541+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=D0UIXZLX4863 tag=RQJPZFMUGB8H type=write size=1250371 2024-03-04T17:50:46.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.545+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=7PQFN3B6E1TB type=del size=1281112 2024-03-04T17:50:46.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.549+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=52ON34M3L7YD type=del 2024-03-04T17:50:46.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.553+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~GUZQFCYQUVXR.0 size=813869 2024-03-04T17:50:46.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.553+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=GUZQFCYQUVXR tag=F5CDSY73WQ59 type=write size=1862445 2024-03-04T17:50:46.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.557+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=SFJJTE3G66JX type=del 2024-03-04T17:50:46.560 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.561+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=KJ2S8A52VSZB type=del size=1476688 2024-03-04T17:50:46.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.565+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~CDX8HQPEPJAN.0 size=1048576 2024-03-04T17:50:46.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.569+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~CDX8HQPEPJAN.1 size=994469 2024-03-04T17:50:46.573 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.573+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=CDX8HQPEPJAN tag=ARCY9NKD20TW type=write size=3091621 2024-03-04T17:50:46.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.577+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=D6ZF0ZQZ0F9W type=del size=1539162 2024-03-04T17:50:46.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.581+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=4MM8YNBT4LS0 type=del 2024-03-04T17:50:46.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.585+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=MHZB1SSQN6PA type=write size=229623 2024-03-04T17:50:46.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.589+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=WSFW311LRXGI type=del 2024-03-04T17:50:46.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.597+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~V7U89QOQN58V.0 size=923008 2024-03-04T17:50:46.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.597+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=V7U89QOQN58V tag=7WOFV02EE9HF type=write size=1971584 2024-03-04T17:50:46.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.601+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=N2KS2T0QUBJ0 type=del 2024-03-04T17:50:46.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.605+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=ZCY5K6KJRD6E type=write size=967249 2024-03-04T17:50:46.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.613+0000 7fda08af98c0 1 < suggested remove operation key=Z2JQBN84QQIB tag=I7HNT3HHKDZK type=del 2024-03-04T17:50:46.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.617+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=6E3WNGL9MSHA type=del size=856353 2024-03-04T17:50:46.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.621+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~3E84ZYDYS6V2.0 size=1048576 2024-03-04T17:50:46.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.625+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~3E84ZYDYS6V2.1 size=1048576 2024-03-04T17:50:46.626 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.629+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~3E84ZYDYS6V2.2 size=723794 2024-03-04T17:50:46.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.633+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=3E84ZYDYS6V2 tag=SYVVPYQF9E8J type=write size=3869522 2024-03-04T17:50:46.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.637+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=1D2POTDP0P3B type=del size=2610872 2024-03-04T17:50:46.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.641+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~JMX6GUTNATKA.0 size=1048576 2024-03-04T17:50:46.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.645+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~JMX6GUTNATKA.1 size=61963 2024-03-04T17:50:46.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.653+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=JMX6GUTNATKA tag=L79TGMPCO0WI type=write size=2159115 2024-03-04T17:50:46.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.657+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~I5UUPCVHPLSJ.0 size=1048576 2024-03-04T17:50:46.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.661+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~I5UUPCVHPLSJ.1 size=396072 2024-03-04T17:50:46.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.661+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=I5UUPCVHPLSJ tag=G7SEJKSODPDU type=write size=2493224 2024-03-04T17:50:46.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.661+0000 7fda08af98c0 1 < canceled part key=_multipart_AO3QS6CM7I5D.2~3E4AQ1S2BI4E.0 size=1048576 2024-03-04T17:50:46.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.665+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~3E4AQ1S2BI4E.1 size=614667 2024-03-04T17:50:46.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.673+0000 7fda08af98c0 1 < canceled multipart upload key=AO3QS6CM7I5D upload=3E4AQ1S2BI4E tag=ZIVGTC9SX2QP type=write size=2711819 2024-03-04T17:50:46.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.677+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=902QN4VEOF72 type=del size=2933530 2024-03-04T17:50:46.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.681+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~6KZWYKOS5V7K.0 size=1048576 2024-03-04T17:50:46.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.685+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~6KZWYKOS5V7K.1 size=638680 2024-03-04T17:50:46.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.689+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=6KZWYKOS5V7K tag=SS5R0EWZFRS8 type=write size=2735832 2024-03-04T17:50:46.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.697+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~P772P92N143P.0 size=770465 2024-03-04T17:50:46.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.697+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=P772P92N143P tag=M9UIKRSKWXJC type=write size=1819041 2024-03-04T17:50:46.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.701+0000 7fda08af98c0 1 < completed write operation key=AO3QS6CM7I5D tag=KMGG7OQD1TYQ type=write size=114613 2024-03-04T17:50:46.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.709+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~61E6V94CC06F.0 size=273305 2024-03-04T17:50:46.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.709+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=61E6V94CC06F tag=Z1ETC2K1PTGP type=write size=1321881 2024-03-04T17:50:46.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.713+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~9G7OZV1HJ5G2.0 size=1048576 2024-03-04T17:50:46.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.713+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~9G7OZV1HJ5G2.1 size=1048576 2024-03-04T17:50:46.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.713+0000 7fda08af98c0 1 < canceled part key=_multipart_3UWSL7NJLACY.2~9G7OZV1HJ5G2.2 size=545897 2024-03-04T17:50:46.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.721+0000 7fda08af98c0 1 < canceled multipart upload key=3UWSL7NJLACY upload=9G7OZV1HJ5G2 tag=P6ZW80OVJ9OJ type=write size=3691625 2024-03-04T17:50:46.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.725+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=Z6B13GZDHYBQ type=del size=3399157 2024-03-04T17:50:46.728 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.729+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~7TQ55WEGQX1J.0 size=1048576 2024-03-04T17:50:46.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.733+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~7TQ55WEGQX1J.1 size=870545 2024-03-04T17:50:46.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.737+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=7TQ55WEGQX1J tag=0MDC20WNEET4 type=write size=2967697 2024-03-04T17:50:46.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.741+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=OLQY750B0SNU type=del size=621683 2024-03-04T17:50:46.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.749+0000 7fda08af98c0 1 < suggested remove operation key=ITIVF3UAT9NK tag=4W06I87Q0QLW type=del 2024-03-04T17:50:46.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.753+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=OX14WA8PA38L type=del size=3883879 2024-03-04T17:50:46.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.757+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=7PQFN3B6E1TB type=del 2024-03-04T17:50:46.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.761+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=ZQYEM8YHHTXM type=del size=2316537 2024-03-04T17:50:46.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.765+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~GUZQFCYQUVXR.0 size=813869 2024-03-04T17:50:46.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.773+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=GUZQFCYQUVXR tag=F5CDSY73WQ59 type=write size=1862445 2024-03-04T17:50:46.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.781+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~XZJ601KPWSX7.0 size=1048576 2024-03-04T17:50:46.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.785+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~XZJ601KPWSX7.1 size=1048576 2024-03-04T17:50:46.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.785+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~XZJ601KPWSX7.2 size=754860 2024-03-04T17:50:46.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.785+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=XZJ601KPWSX7 tag=IULG8PNEXE0Z type=write size=3900588 2024-03-04T17:50:46.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.789+0000 7fda08af98c0 -1 < failed to complete operation key=Z2JQBN84QQIB tag=KJ2S8A52VSZB type=del size=1476688: (22) Invalid argument 2024-03-04T17:50:46.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.793+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=VOQ8LBFPWJB9 type=del size=1656622 2024-03-04T17:50:46.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.801+0000 7fda08af98c0 1 < suggested remove operation key=HCA7QLGM2O4O tag=D6ZF0ZQZ0F9W type=del 2024-03-04T17:50:46.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.809+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~8S096K9FMOG6.0 size=352148 2024-03-04T17:50:46.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.809+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=8S096K9FMOG6 tag=SBZE2OO9RMO9 type=write size=1400724 2024-03-04T17:50:46.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.813+0000 7fda08af98c0 1 < completed write operation key=AO3QS6CM7I5D tag=MHZB1SSQN6PA type=write size=229623 2024-03-04T17:50:46.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.817+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=WW774K2HZ93O type=del size=2345139 2024-03-04T17:50:46.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.825+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~V7U89QOQN58V.0 size=923008 2024-03-04T17:50:46.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.829+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=V7U89QOQN58V tag=7WOFV02EE9HF type=write size=1971584 2024-03-04T17:50:46.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.833+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=30KTSFHAG8DA type=del size=1512856 2024-03-04T17:50:46.837 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.841+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=ZCY5K6KJRD6E type=write size=967249 2024-03-04T17:50:46.844 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.845+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~UG4I1IMX6FB0.0 size=886455 2024-03-04T17:50:46.844 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.845+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=UG4I1IMX6FB0 tag=4R1B5LI3WO1Y type=write size=1935031 2024-03-04T17:50:46.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.849+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=6E3WNGL9MSHA type=del 2024-03-04T17:50:46.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.857+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=12I31YZF3UP1 type=del size=239912 2024-03-04T17:50:46.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.861+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=1D2POTDP0P3B type=del 2024-03-04T17:50:46.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.869+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~S8AMI4EAUIHD.0 size=1048576 2024-03-04T17:50:46.869 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.873+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~S8AMI4EAUIHD.1 size=1048576 2024-03-04T17:50:46.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.873+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~S8AMI4EAUIHD.2 size=478224 2024-03-04T17:50:46.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.873+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=S8AMI4EAUIHD tag=7TN5Z5GLGE50 type=write size=3623952 2024-03-04T17:50:46.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.881+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~I5UUPCVHPLSJ.0 size=1048576 2024-03-04T17:50:46.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.881+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~I5UUPCVHPLSJ.1 size=396072 2024-03-04T17:50:46.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.889+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=I5UUPCVHPLSJ tag=G7SEJKSODPDU type=write size=2493224 2024-03-04T17:50:46.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.893+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=4WJ86WHC5OOE type=del size=1885372 2024-03-04T17:50:46.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.901+0000 7fda08af98c0 1 < suggested update operation key=2PA8JDZUOLO5 tag=902QN4VEOF72 type=del size=2933530 2024-03-04T17:50:46.903 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.905+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=8KOVM5BQSLZU type=write size=257610 2024-03-04T17:50:46.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.909+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~P772P92N143P.0 size=770465 2024-03-04T17:50:46.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.913+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=P772P92N143P tag=M9UIKRSKWXJC type=write size=1819041 2024-03-04T17:50:46.920 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.921+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~FXE3IUMVL0QC.0 size=1048576 2024-03-04T17:50:46.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.925+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~FXE3IUMVL0QC.1 size=1048576 2024-03-04T17:50:46.925 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.929+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~FXE3IUMVL0QC.2 size=553608 2024-03-04T17:50:46.925 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.929+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=FXE3IUMVL0QC tag=6MAWPBXXFLSA type=write size=3699336 2024-03-04T17:50:46.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.933+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~61E6V94CC06F.0 size=273305 2024-03-04T17:50:46.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.937+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=61E6V94CC06F tag=Z1ETC2K1PTGP type=write size=1321881 2024-03-04T17:50:46.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.945+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~RH37BGSH87EO.0 size=1048576 2024-03-04T17:50:46.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.945+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~RH37BGSH87EO.1 size=1048576 2024-03-04T17:50:46.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.949+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~RH37BGSH87EO.2 size=558147 2024-03-04T17:50:46.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.949+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=RH37BGSH87EO tag=114FDW06684W type=write size=3703875 2024-03-04T17:50:46.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.953+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=Z6B13GZDHYBQ type=del 2024-03-04T17:50:46.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.961+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~W3BTNL2SZQ7T.0 size=959618 2024-03-04T17:50:46.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.961+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=W3BTNL2SZQ7T tag=8LTKMSSRYGO0 type=write size=2008194 2024-03-04T17:50:46.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.965+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=OLQY750B0SNU type=del 2024-03-04T17:50:46.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.969+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=8ZE5199TVWLG type=del size=1182039 2024-03-04T17:50:46.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.977+0000 7fda08af98c0 1 < suggested remove operation key=RXVSQVVNXVF0 tag=OX14WA8PA38L type=del 2024-03-04T17:50:46.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.981+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~4XHUPVSC1WDZ.0 size=494941 2024-03-04T17:50:46.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.981+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=4XHUPVSC1WDZ tag=OF0NC80YQM6L type=write size=1543517 2024-03-04T17:50:46.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.985+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=ZQYEM8YHHTXM type=del 2024-03-04T17:50:46.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.989+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=S5SWAKTY26L9 type=del size=3208906 2024-03-04T17:50:46.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.993+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~XZJ601KPWSX7.0 size=1048576 2024-03-04T17:50:46.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.993+0000 7fda08af98c0 1 < canceled part key=_multipart_XG2XYBCMUNY5.2~XZJ601KPWSX7.1 size=1048576 2024-03-04T17:50:46.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:46.997+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~XZJ601KPWSX7.2 size=754860 2024-03-04T17:50:47.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.001+0000 7fda08af98c0 1 < canceled multipart upload key=XG2XYBCMUNY5 upload=XZJ601KPWSX7 tag=IULG8PNEXE0Z type=write size=3900588 2024-03-04T17:50:47.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.005+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=K64GW3LUSUOZ type=del size=345800 2024-03-04T17:50:47.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.009+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=VOQ8LBFPWJB9 type=del 2024-03-04T17:50:47.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.013+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=2CGTYY3CIESJ type=del size=4167897 2024-03-04T17:50:47.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.017+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~8S096K9FMOG6.0 size=352148 2024-03-04T17:50:47.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.025+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=8S096K9FMOG6 tag=SBZE2OO9RMO9 type=write size=1400724 2024-03-04T17:50:47.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.029+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=8SKIBQJ8BN40 type=write size=532599 2024-03-04T17:50:47.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.033+0000 7fda08af98c0 -1 < failed to complete operation key=2PA8JDZUOLO5 tag=WW774K2HZ93O type=del size=2345139: (22) Invalid argument 2024-03-04T17:50:47.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.037+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=L9F2MO6PFALQ type=write size=668938 2024-03-04T17:50:47.042 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.045+0000 7fda08af98c0 1 < suggested remove operation key=AO3QS6CM7I5D tag=30KTSFHAG8DA type=del 2024-03-04T17:50:47.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.049+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~BI8A40VI6HM3.0 size=1048576 2024-03-04T17:50:47.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.053+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~BI8A40VI6HM3.1 size=647373 2024-03-04T17:50:47.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.053+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=BI8A40VI6HM3 tag=CN1064Q7NM4S type=write size=2744525 2024-03-04T17:50:47.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.057+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~UG4I1IMX6FB0.0 size=886455 2024-03-04T17:50:47.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.065+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=UG4I1IMX6FB0 tag=4R1B5LI3WO1Y type=write size=1935031 2024-03-04T17:50:47.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.069+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=E96TEMUXJ019 type=write size=650160 2024-03-04T17:50:47.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.077+0000 7fda08af98c0 1 < suggested update operation key=BYQFZSJ3WH07 tag=12I31YZF3UP1 type=del size=239912 2024-03-04T17:50:47.078 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.081+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=SLNAEUA6FC2N type=del size=3207539 2024-03-04T17:50:47.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.085+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~S8AMI4EAUIHD.0 size=1048576 2024-03-04T17:50:47.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.085+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~S8AMI4EAUIHD.1 size=1048576 2024-03-04T17:50:47.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.089+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~S8AMI4EAUIHD.2 size=478224 2024-03-04T17:50:47.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.097+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=S8AMI4EAUIHD tag=7TN5Z5GLGE50 type=write size=3623952 2024-03-04T17:50:47.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.101+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=U9YV7EPNZRJO type=del size=448043 2024-03-04T17:50:47.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.105+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=4WJ86WHC5OOE type=del 2024-03-04T17:50:47.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.109+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=VZARJDBIDCAQ type=del size=3241680 2024-03-04T17:50:47.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.113+0000 7fda08af98c0 1 < completed write operation key=XG5I3SDRE5JM tag=8KOVM5BQSLZU type=write size=257610 2024-03-04T17:50:47.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.117+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=SJMU3H94GLG7 type=del size=4134288 2024-03-04T17:50:47.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.121+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~FXE3IUMVL0QC.0 size=1048576 2024-03-04T17:50:47.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.121+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~FXE3IUMVL0QC.1 size=1048576 2024-03-04T17:50:47.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.125+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~FXE3IUMVL0QC.2 size=553608 2024-03-04T17:50:47.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.133+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=FXE3IUMVL0QC tag=6MAWPBXXFLSA type=write size=3699336 2024-03-04T17:50:47.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.141+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~XOGY5JJXU1BG.0 size=1048576 2024-03-04T17:50:47.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.141+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~XOGY5JJXU1BG.1 size=1048576 2024-03-04T17:50:47.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.145+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~XOGY5JJXU1BG.2 size=958684 2024-03-04T17:50:47.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.145+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=XOGY5JJXU1BG tag=XYMWZECSVFZM type=write size=4104412 2024-03-04T17:50:47.148 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.149+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~RH37BGSH87EO.0 size=1048576 2024-03-04T17:50:47.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.153+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~RH37BGSH87EO.1 size=1048576 2024-03-04T17:50:47.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.157+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~RH37BGSH87EO.2 size=558147 2024-03-04T17:50:47.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.161+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=RH37BGSH87EO tag=114FDW06684W type=write size=3703875 2024-03-04T17:50:47.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.169+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~VOEZOL2HL6N4.0 size=1048576 2024-03-04T17:50:47.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.173+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~VOEZOL2HL6N4.1 size=1048576 2024-03-04T17:50:47.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.177+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~VOEZOL2HL6N4.2 size=795232 2024-03-04T17:50:47.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.177+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=VOEZOL2HL6N4 tag=F3K1YADTOYJL type=write size=3940960 2024-03-04T17:50:47.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.185+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~W3BTNL2SZQ7T.0 size=959618 2024-03-04T17:50:47.190 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.193+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=W3BTNL2SZQ7T tag=8LTKMSSRYGO0 type=write size=2008194 2024-03-04T17:50:47.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.197+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=474KFQLU0Q47 type=del size=3976040 2024-03-04T17:50:47.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.205+0000 7fda08af98c0 1 < suggested remove operation key=FMPZ2548LBBF tag=8ZE5199TVWLG type=del 2024-03-04T17:50:47.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.209+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~OJXOVAJ34HNY.0 size=1048576 2024-03-04T17:50:47.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.213+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~OJXOVAJ34HNY.1 size=938637 2024-03-04T17:50:47.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.213+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=OJXOVAJ34HNY tag=M7CIGBRHREOJ type=write size=3035789 2024-03-04T17:50:47.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.217+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~4XHUPVSC1WDZ.0 size=494941 2024-03-04T17:50:47.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.225+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=4XHUPVSC1WDZ tag=OF0NC80YQM6L type=write size=1543517 2024-03-04T17:50:47.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.229+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=8R864MYRKIH2 type=del size=319753 2024-03-04T17:50:47.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.233+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=S5SWAKTY26L9 type=del 2024-03-04T17:50:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.237+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=52E8RRFDP59D type=del size=2776876 2024-03-04T17:50:47.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.241+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=K64GW3LUSUOZ type=del 2024-03-04T17:50:47.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.245+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=6PQW8VGFPKL6 type=del size=1662929 2024-03-04T17:50:47.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.249+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=2CGTYY3CIESJ type=del 2024-03-04T17:50:47.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.253+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=6EEZ15V7760N type=del size=3159352 2024-03-04T17:50:47.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.261+0000 7fda08af98c0 1 < suggested update operation key=V1LDB2HXZ7EF tag=8SKIBQJ8BN40 type=write size=532599 2024-03-04T17:50:47.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.269+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~6XFO6MFCHGTF.0 size=1048576 2024-03-04T17:50:47.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.273+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~6XFO6MFCHGTF.1 size=1048576 2024-03-04T17:50:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.277+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~6XFO6MFCHGTF.2 size=434736 2024-03-04T17:50:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.277+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=6XFO6MFCHGTF tag=MXH2FKNQJQGC type=write size=3580464 2024-03-04T17:50:47.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.285+0000 7fda08af98c0 1 < suggested remove operation key=1OZKPHMPV3OQ tag=L9F2MO6PFALQ type=write 2024-03-04T17:50:47.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.293+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~IYM6X1W3H05N.0 size=1048576 2024-03-04T17:50:47.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.297+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~IYM6X1W3H05N.1 size=891816 2024-03-04T17:50:47.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.297+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=IYM6X1W3H05N tag=YYOL6CPHDTVE type=write size=2988968 2024-03-04T17:50:47.299 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.301+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~BI8A40VI6HM3.0 size=1048576 2024-03-04T17:50:47.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.305+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~BI8A40VI6HM3.1 size=647373 2024-03-04T17:50:47.308 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.309+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=BI8A40VI6HM3 tag=CN1064Q7NM4S type=write size=2744525 2024-03-04T17:50:47.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.317+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~4RH0VPA5G0RI.0 size=1048576 2024-03-04T17:50:47.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.321+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~4RH0VPA5G0RI.1 size=66222 2024-03-04T17:50:47.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.321+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=4RH0VPA5G0RI tag=1IBE0EOILN9X type=write size=2163374 2024-03-04T17:50:47.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.325+0000 7fda08af98c0 1 < completed write operation key=IJSYAVR5SWPS tag=E96TEMUXJ019 type=write size=650160 2024-03-04T17:50:47.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.325+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=FRLC3U21O8PP type=del size=1083078 2024-03-04T17:50:47.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.329+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=SLNAEUA6FC2N type=del 2024-03-04T17:50:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.333+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=805AI6HC3NPG type=del size=900996 2024-03-04T17:50:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.337+0000 7fda08af98c0 -1 < failed to complete operation key=FMPZ2548LBBF tag=U9YV7EPNZRJO type=del size=448043: (22) Invalid argument 2024-03-04T17:50:47.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.341+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=Q51MWH24QPKR type=del size=2690009 2024-03-04T17:50:47.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.345+0000 7fda08af98c0 -1 < failed to complete operation key=V1LDB2HXZ7EF tag=VZARJDBIDCAQ type=del size=3241680: (22) Invalid argument 2024-03-04T17:50:47.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.349+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=QBH54S72YPMV type=del size=1352659 2024-03-04T17:50:47.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.353+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=SJMU3H94GLG7 type=del 2024-03-04T17:50:47.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.361+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~AI7UT7HNZNZW.0 size=547856 2024-03-04T17:50:47.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.361+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=AI7UT7HNZNZW tag=4JR1AYOYF7ML type=write size=1596432 2024-03-04T17:50:47.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.365+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~XOGY5JJXU1BG.0 size=1048576 2024-03-04T17:50:47.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.369+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~XOGY5JJXU1BG.1 size=1048576 2024-03-04T17:50:47.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.369+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~XOGY5JJXU1BG.2 size=958684 2024-03-04T17:50:47.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.377+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=XOGY5JJXU1BG tag=XYMWZECSVFZM type=write size=4104412 2024-03-04T17:50:47.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.385+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~DMMK6PQN6AM3.0 size=687686 2024-03-04T17:50:47.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.385+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=DMMK6PQN6AM3 tag=B5GD3TYSESH6 type=write size=1736262 2024-03-04T17:50:47.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.385+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~VOEZOL2HL6N4.0 size=1048576 2024-03-04T17:50:47.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.389+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~VOEZOL2HL6N4.1 size=1048576 2024-03-04T17:50:47.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.389+0000 7fda08af98c0 1 < canceled part key=_multipart_RXVSQVVNXVF0.2~VOEZOL2HL6N4.2 size=795232 2024-03-04T17:50:47.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.397+0000 7fda08af98c0 1 < canceled multipart upload key=RXVSQVVNXVF0 upload=VOEZOL2HL6N4 tag=F3K1YADTOYJL type=write size=3940960 2024-03-04T17:50:47.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.401+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=CYF5GIQ14POA type=del size=1774544 2024-03-04T17:50:47.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.405+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=474KFQLU0Q47 type=del 2024-03-04T17:50:47.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.409+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=02PIQ8QHZS7J type=del size=2044737 2024-03-04T17:50:47.410 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.413+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~OJXOVAJ34HNY.0 size=1048576 2024-03-04T17:50:47.414 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.413+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~OJXOVAJ34HNY.1 size=938637 2024-03-04T17:50:47.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.421+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=OJXOVAJ34HNY tag=M7CIGBRHREOJ type=write size=3035789 2024-03-04T17:50:47.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.425+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=B7TC0V1DHGBM type=del size=3475634 2024-03-04T17:50:47.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.429+0000 7fda08af98c0 1 < canceled operation key=XG5I3SDRE5JM tag=8R864MYRKIH2 type=del size=319753 2024-03-04T17:50:47.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.433+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=KWNRCS2MCDDK type=write size=513117 2024-03-04T17:50:47.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.437+0000 7fda08af98c0 -1 < failed to complete operation key=1OZKPHMPV3OQ tag=52E8RRFDP59D type=del size=2776876: (22) Invalid argument 2024-03-04T17:50:47.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.445+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~K84Q1VPQ887O.0 size=456967 2024-03-04T17:50:47.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.445+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=K84Q1VPQ887O tag=CUYF513HJ883 type=write size=1505543 2024-03-04T17:50:47.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.449+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=6PQW8VGFPKL6 type=del size=1662929 2024-03-04T17:50:47.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.453+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~1WCL31ES3301.0 size=342233 2024-03-04T17:50:47.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.453+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=1WCL31ES3301 tag=6IQW8LR61IYK type=write size=1390809 2024-03-04T17:50:47.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.461+0000 7fda08af98c0 1 < suggested remove operation key=2PA8JDZUOLO5 tag=6EEZ15V7760N type=del 2024-03-04T17:50:47.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.469+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~WGFV1A6WTP9U.0 size=1048576 2024-03-04T17:50:47.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.473+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~WGFV1A6WTP9U.1 size=1048576 2024-03-04T17:50:47.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.477+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~WGFV1A6WTP9U.2 size=306651 2024-03-04T17:50:47.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.477+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=WGFV1A6WTP9U tag=QLTSR84JTFG3 type=write size=3452379 2024-03-04T17:50:47.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.477+0000 7fda08af98c0 1 < canceled part key=_multipart_AO3QS6CM7I5D.2~6XFO6MFCHGTF.0 size=1048576 2024-03-04T17:50:47.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.481+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~6XFO6MFCHGTF.1 size=1048576 2024-03-04T17:50:47.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.481+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~6XFO6MFCHGTF.2 size=434736 2024-03-04T17:50:47.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.489+0000 7fda08af98c0 1 < canceled multipart upload key=AO3QS6CM7I5D upload=6XFO6MFCHGTF tag=MXH2FKNQJQGC type=write size=3580464 2024-03-04T17:50:47.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.493+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=XXLPNZOB79TG type=write size=357072 2024-03-04T17:50:47.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.493+0000 7fda08af98c0 1 < canceled part key=_multipart_Z3J7W9V8HQKH.2~IYM6X1W3H05N.0 size=1048576 2024-03-04T17:50:47.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.497+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~IYM6X1W3H05N.1 size=891816 2024-03-04T17:50:47.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.501+0000 7fda08af98c0 1 < canceled multipart upload key=Z3J7W9V8HQKH upload=IYM6X1W3H05N tag=YYOL6CPHDTVE type=write size=2988968 2024-03-04T17:50:47.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.505+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=93SM9GIXTT73 type=del size=1834157 2024-03-04T17:50:47.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.509+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~4RH0VPA5G0RI.0 size=1048576 2024-03-04T17:50:47.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.513+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~4RH0VPA5G0RI.1 size=66222 2024-03-04T17:50:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.517+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=4RH0VPA5G0RI tag=1IBE0EOILN9X type=write size=2163374 2024-03-04T17:50:47.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.521+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=BMI2772PELCD type=write size=60199 2024-03-04T17:50:47.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.525+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=FRLC3U21O8PP type=del 2024-03-04T17:50:47.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.533+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~NY0E3NBF2ZYW.0 size=1048576 2024-03-04T17:50:47.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.537+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~NY0E3NBF2ZYW.1 size=313317 2024-03-04T17:50:47.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.537+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=NY0E3NBF2ZYW tag=SWF0TWWCV9MB type=write size=2410469 2024-03-04T17:50:47.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.549+0000 7fda08af98c0 1 < suggested update operation key=AO3QS6CM7I5D tag=805AI6HC3NPG type=del size=900996 2024-03-04T17:50:47.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.557+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~2QBV7QK8T6IW.0 size=1048576 2024-03-04T17:50:47.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.557+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~2QBV7QK8T6IW.1 size=975160 2024-03-04T17:50:47.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.557+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=2QBV7QK8T6IW tag=N1T9CB3W28KE type=write size=3072312 2024-03-04T17:50:47.560 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.561+0000 7fda08af98c0 1 < canceled operation key=X5MEJ7G8X9HE tag=Q51MWH24QPKR type=del size=2690009 2024-03-04T17:50:47.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.565+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=48IIPJIB00DN type=del size=189942 2024-03-04T17:50:47.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.569+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=QBH54S72YPMV type=del 2024-03-04T17:50:47.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.573+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=N0R95OD2O529 type=del size=282947 2024-03-04T17:50:47.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.577+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~AI7UT7HNZNZW.0 size=547856 2024-03-04T17:50:47.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.585+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=AI7UT7HNZNZW tag=4JR1AYOYF7ML type=write size=1596432 2024-03-04T17:50:47.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.593+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~AWEA1II5T9R7.0 size=1048576 2024-03-04T17:50:47.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.597+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~AWEA1II5T9R7.1 size=603172 2024-03-04T17:50:47.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.597+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=AWEA1II5T9R7 tag=43BYVFX7KSNA type=write size=2700324 2024-03-04T17:50:47.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.601+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~DMMK6PQN6AM3.0 size=687686 2024-03-04T17:50:47.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.609+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=DMMK6PQN6AM3 tag=B5GD3TYSESH6 type=write size=1736262 2024-03-04T17:50:47.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.613+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=LVHCY7WM74PS type=del size=3387334 2024-03-04T17:50:47.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.617+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=CYF5GIQ14POA type=del 2024-03-04T17:50:47.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.625+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~K104RR5H6Y80.0 size=1048576 2024-03-04T17:50:47.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.629+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~K104RR5H6Y80.1 size=266313 2024-03-04T17:50:47.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.629+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=K104RR5H6Y80 tag=KB81L1T7R33Y type=write size=2363465 2024-03-04T17:50:47.635 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.637+0000 7fda08af98c0 1 < suggested update operation key=HCA7QLGM2O4O tag=02PIQ8QHZS7J type=del size=2044737 2024-03-04T17:50:47.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.645+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~I5L1OW1IQFBV.0 size=1014471 2024-03-04T17:50:47.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.645+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=I5L1OW1IQFBV tag=PTIOLAMZ7FXJ type=write size=2063047 2024-03-04T17:50:47.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.649+0000 7fda08af98c0 -1 < failed to complete operation key=2PA8JDZUOLO5 tag=B7TC0V1DHGBM type=del size=3475634: (22) Invalid argument 2024-03-04T17:50:47.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.653+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~OJACIDY9JF63.0 size=1048576 2024-03-04T17:50:47.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.657+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~OJACIDY9JF63.1 size=40084 2024-03-04T17:50:47.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.657+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=OJACIDY9JF63 tag=H301PN795WTV type=write size=2137236 2024-03-04T17:50:47.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.661+0000 7fda08af98c0 1 < completed write operation key=HCKJZUMAK0Z0 tag=KWNRCS2MCDDK type=write size=513117 2024-03-04T17:50:47.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.665+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=P3KSU5FSL2L2 type=del size=4093673 2024-03-04T17:50:47.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.669+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~K84Q1VPQ887O.0 size=456967 2024-03-04T17:50:47.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.673+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=K84Q1VPQ887O tag=CUYF513HJ883 type=write size=1505543 2024-03-04T17:50:47.677 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.677+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=6WH9LBMMK1SY type=del size=1562704 2024-03-04T17:50:47.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.681+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~1WCL31ES3301.0 size=342233 2024-03-04T17:50:47.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.689+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=1WCL31ES3301 tag=6IQW8LR61IYK type=write size=1390809 2024-03-04T17:50:47.690 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.693+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=I1JX4EVVFCHX type=del size=499548 2024-03-04T17:50:47.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.697+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~WGFV1A6WTP9U.0 size=1048576 2024-03-04T17:50:47.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.697+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~WGFV1A6WTP9U.1 size=1048576 2024-03-04T17:50:47.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.701+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~WGFV1A6WTP9U.2 size=306651 2024-03-04T17:50:47.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.705+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=WGFV1A6WTP9U tag=QLTSR84JTFG3 type=write size=3452379 2024-03-04T17:50:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.709+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=95TJ90JKMO8Q type=del size=1545965 2024-03-04T17:50:47.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.713+0000 7fda08af98c0 1 < completed write operation key=BYQFZSJ3WH07 tag=XXLPNZOB79TG type=write size=357072 2024-03-04T17:50:47.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.717+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=V38QJHGD66C8 type=del size=775706 2024-03-04T17:50:47.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.725+0000 7fda08af98c0 1 < suggested remove operation key=8YL7C9PI58TK tag=93SM9GIXTT73 type=del 2024-03-04T17:50:47.729 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.729+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=QHHNOL67LIRX type=del size=1340841 2024-03-04T17:50:47.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.733+0000 7fda08af98c0 -1 < failed to complete operation key=HCA7QLGM2O4O tag=BMI2772PELCD type=write size=60199: (22) Invalid argument 2024-03-04T17:50:47.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.737+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=TZU7C2X81HOB type=del size=3796238 2024-03-04T17:50:47.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.745+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~NY0E3NBF2ZYW.0 size=1048576 2024-03-04T17:50:47.744 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.745+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~NY0E3NBF2ZYW.1 size=313317 2024-03-04T17:50:47.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.753+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=NY0E3NBF2ZYW tag=SWF0TWWCV9MB type=write size=2410469 2024-03-04T17:50:47.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.761+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~G5R5MCUXEGAA.0 size=1048576 2024-03-04T17:50:47.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.761+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~G5R5MCUXEGAA.1 size=456342 2024-03-04T17:50:47.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.761+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=G5R5MCUXEGAA tag=ZR7FI6XBEH4X type=write size=2553494 2024-03-04T17:50:47.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.769+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~2QBV7QK8T6IW.0 size=1048576 2024-03-04T17:50:47.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.769+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~2QBV7QK8T6IW.1 size=975160 2024-03-04T17:50:47.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.777+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=2QBV7QK8T6IW tag=N1T9CB3W28KE type=write size=3072312 2024-03-04T17:50:47.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.781+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=YXMO0YE283V6 type=write size=996833 2024-03-04T17:50:47.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.785+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=48IIPJIB00DN type=del 2024-03-04T17:50:47.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.789+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=F8L8X9Q0PTYP type=del size=2186861 2024-03-04T17:50:47.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.793+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=N0R95OD2O529 type=del 2024-03-04T17:50:47.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.801+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~0RZM1GRJJ7VQ.0 size=1048576 2024-03-04T17:50:47.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.801+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~0RZM1GRJJ7VQ.1 size=1048576 2024-03-04T17:50:47.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.805+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~0RZM1GRJJ7VQ.2 size=762812 2024-03-04T17:50:47.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.805+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=0RZM1GRJJ7VQ tag=LJKR47HU1HJG type=write size=3908540 2024-03-04T17:50:47.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.809+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~AWEA1II5T9R7.0 size=1048576 2024-03-04T17:50:47.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.813+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~AWEA1II5T9R7.1 size=603172 2024-03-04T17:50:47.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.821+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=AWEA1II5T9R7 tag=43BYVFX7KSNA type=write size=2700324 2024-03-04T17:50:47.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.825+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=7JZVP6ZKUYM9 type=del size=2897001 2024-03-04T17:50:47.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.829+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=LVHCY7WM74PS type=del 2024-03-04T17:50:47.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.833+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~Y3KC8OLH2T72.0 size=1048576 2024-03-04T17:50:47.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.837+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~Y3KC8OLH2T72.1 size=1048576 2024-03-04T17:50:47.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.841+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~Y3KC8OLH2T72.2 size=347893 2024-03-04T17:50:47.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.841+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=Y3KC8OLH2T72 tag=5AODOIID3TSE type=write size=3493621 2024-03-04T17:50:47.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.845+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~K104RR5H6Y80.0 size=1048576 2024-03-04T17:50:47.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.849+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~K104RR5H6Y80.1 size=266313 2024-03-04T17:50:47.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.853+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=K104RR5H6Y80 tag=KB81L1T7R33Y type=write size=2363465 2024-03-04T17:50:47.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.857+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=WOROBQ0MG5E8 type=del size=3581240 2024-03-04T17:50:47.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.861+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~I5L1OW1IQFBV.0 size=1014471 2024-03-04T17:50:47.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.869+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=I5L1OW1IQFBV tag=PTIOLAMZ7FXJ type=write size=2063047 2024-03-04T17:50:47.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.873+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=BDKYC2WDER0R type=del size=828571 2024-03-04T17:50:47.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.877+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~OJACIDY9JF63.0 size=1048576 2024-03-04T17:50:47.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.877+0000 7fda08af98c0 1 < canceled part key=_multipart_FNWA2N9U14HO.2~OJACIDY9JF63.1 size=40084 2024-03-04T17:50:47.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.881+0000 7fda08af98c0 1 < canceled multipart upload key=FNWA2N9U14HO upload=OJACIDY9JF63 tag=H301PN795WTV type=write size=2137236 2024-03-04T17:50:47.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.885+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=4XAWNIZRFX03 type=del size=2222714 2024-03-04T17:50:47.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.893+0000 7fda08af98c0 1 < suggested remove operation key=0IGSMFORCSL9 tag=P3KSU5FSL2L2 type=del 2024-03-04T17:50:47.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.901+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~JADNJFS163E0.0 size=1048576 2024-03-04T17:50:47.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.905+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~JADNJFS163E0.1 size=1048576 2024-03-04T17:50:47.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.913+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~JADNJFS163E0.2 size=788633 2024-03-04T17:50:47.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.913+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=JADNJFS163E0 tag=7WNWBZBZ1PT6 type=write size=3934361 2024-03-04T17:50:47.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.917+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=6WH9LBMMK1SY type=del 2024-03-04T17:50:47.923 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.925+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~UP4AG03OQATA.0 size=1048576 2024-03-04T17:50:47.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.929+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~UP4AG03OQATA.1 size=596266 2024-03-04T17:50:47.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.929+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=UP4AG03OQATA tag=NPCPP8NW99ZO type=write size=2693418 2024-03-04T17:50:47.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.933+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=I1JX4EVVFCHX type=del 2024-03-04T17:50:47.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.937+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=NZ3ES49EI5QS type=del size=3547242 2024-03-04T17:50:47.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.941+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=95TJ90JKMO8Q type=del 2024-03-04T17:50:47.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.945+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=4QS95ZJ4BC4Y type=del size=1781474 2024-03-04T17:50:47.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.949+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=V38QJHGD66C8 type=del 2024-03-04T17:50:47.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.953+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=TYM7XKQF80QP type=del size=1498313 2024-03-04T17:50:47.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.961+0000 7fda08af98c0 1 < suggested update operation key=750NSVFBS03A tag=QHHNOL67LIRX type=del size=1340841 2024-03-04T17:50:47.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.965+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=QJXVHB86EIT6 type=del size=1794868 2024-03-04T17:50:47.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.973+0000 7fda08af98c0 1 < suggested update operation key=ITIVF3UAT9NK tag=TZU7C2X81HOB type=del size=3796238 2024-03-04T17:50:47.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.973+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=O8GZP4LRUP1P type=del size=4184868 2024-03-04T17:50:47.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.977+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~G5R5MCUXEGAA.0 size=1048576 2024-03-04T17:50:47.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.981+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~G5R5MCUXEGAA.1 size=456342 2024-03-04T17:50:47.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.989+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=G5R5MCUXEGAA tag=ZR7FI6XBEH4X type=write size=2553494 2024-03-04T17:50:47.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.993+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=DQ3AX2VSGYZM type=del size=3697783 2024-03-04T17:50:47.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:47.997+0000 7fda08af98c0 1 < completed write operation key=AO3QS6CM7I5D tag=YXMO0YE283V6 type=write size=996833 2024-03-04T17:50:47.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.001+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=BQ6GX7QBLPA1 type=del size=3037102 2024-03-04T17:50:48.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.005+0000 7fda08af98c0 1 < suggested remove operation key=VJVORUG95FLC tag=F8L8X9Q0PTYP type=del 2024-03-04T17:50:48.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.013+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~3YTL0SYO3S39.0 size=1048576 2024-03-04T17:50:48.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.017+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~3YTL0SYO3S39.1 size=1048576 2024-03-04T17:50:48.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.017+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~3YTL0SYO3S39.2 size=797022 2024-03-04T17:50:48.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.017+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=3YTL0SYO3S39 tag=E4YTNUJZW539 type=write size=3942750 2024-03-04T17:50:48.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.021+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~0RZM1GRJJ7VQ.0 size=1048576 2024-03-04T17:50:48.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.025+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~0RZM1GRJJ7VQ.1 size=1048576 2024-03-04T17:50:48.027 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.029+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~0RZM1GRJJ7VQ.2 size=762812 2024-03-04T17:50:48.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.037+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=0RZM1GRJJ7VQ tag=LJKR47HU1HJG type=write size=3908540 2024-03-04T17:50:48.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.041+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=704M7SLWXP40 type=del size=3164475 2024-03-04T17:50:48.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.045+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=7JZVP6ZKUYM9 type=del 2024-03-04T17:50:48.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.049+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=19EC44I8443Y type=write size=359593 2024-03-04T17:50:48.051 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.053+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~Y3KC8OLH2T72.0 size=1048576 2024-03-04T17:50:48.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.057+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~Y3KC8OLH2T72.1 size=1048576 2024-03-04T17:50:48.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.057+0000 7fda08af98c0 1 < canceled part key=_multipart_L0FVKWD7EQRW.2~Y3KC8OLH2T72.2 size=347893 2024-03-04T17:50:48.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.065+0000 7fda08af98c0 1 < canceled multipart upload key=L0FVKWD7EQRW upload=Y3KC8OLH2T72 tag=5AODOIID3TSE type=write size=3493621 2024-03-04T17:50:48.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.069+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=79FYHVN80SFI type=del size=3794080 2024-03-04T17:50:48.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.069+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=WOROBQ0MG5E8 type=del 2024-03-04T17:50:48.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.073+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=LRDTES2F0QQN type=write size=251448 2024-03-04T17:50:48.077 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.077+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=BDKYC2WDER0R type=del 2024-03-04T17:50:48.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.081+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=4RWG4JLG4GSE type=del size=3784314 2024-03-04T17:50:48.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.085+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=4XAWNIZRFX03 type=del 2024-03-04T17:50:48.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.089+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=THH25Y7XGS5L type=del size=4158385 2024-03-04T17:50:48.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.097+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~JADNJFS163E0.0 size=1048576 2024-03-04T17:50:48.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.097+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~JADNJFS163E0.1 size=1048576 2024-03-04T17:50:48.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.097+0000 7fda08af98c0 1 < canceled part key=_multipart_VJVORUG95FLC.2~JADNJFS163E0.2 size=788633 2024-03-04T17:50:48.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.105+0000 7fda08af98c0 1 < canceled multipart upload key=VJVORUG95FLC upload=JADNJFS163E0 tag=7WNWBZBZ1PT6 type=write size=3934361 2024-03-04T17:50:48.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.109+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=S0V61JJKB6MM type=del size=3749662 2024-03-04T17:50:48.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.113+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~UP4AG03OQATA.0 size=1048576 2024-03-04T17:50:48.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.117+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~UP4AG03OQATA.1 size=596266 2024-03-04T17:50:48.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.125+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=UP4AG03OQATA tag=NPCPP8NW99ZO type=write size=2693418 2024-03-04T17:50:48.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.125+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=AZT5554DISEY type=del size=689598 2024-03-04T17:50:48.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.129+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=NZ3ES49EI5QS type=del 2024-03-04T17:50:48.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.133+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=QFGV7Y13KBQ8 type=del size=1229364 2024-03-04T17:50:48.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.137+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=4QS95ZJ4BC4Y type=del 2024-03-04T17:50:48.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.141+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=3XSYXJK0UPTK type=write size=17882 2024-03-04T17:50:48.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.145+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=TYM7XKQF80QP type=del 2024-03-04T17:50:48.148 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.149+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=UNGGFD2DC1AM type=del size=1488638 2024-03-04T17:50:48.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.153+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=QJXVHB86EIT6 type=del 2024-03-04T17:50:48.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.161+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~CT44ZKOA71SW.0 size=1048576 2024-03-04T17:50:48.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.161+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~CT44ZKOA71SW.1 size=944700 2024-03-04T17:50:48.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.161+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=CT44ZKOA71SW tag=R1414B978WXX type=write size=3041852 2024-03-04T17:50:48.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.173+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=O8GZP4LRUP1P type=del 2024-03-04T17:50:48.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.177+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~H4C0U4CH6XHL.0 size=840424 2024-03-04T17:50:48.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.177+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=H4C0U4CH6XHL tag=VAAILOW2YD5Q type=write size=1889000 2024-03-04T17:50:48.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.181+0000 7fda08af98c0 -1 < failed to complete operation key=VJVORUG95FLC tag=DQ3AX2VSGYZM type=del size=3697783: (22) Invalid argument 2024-03-04T17:50:48.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.185+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=G8M379GFYZEE type=del size=3920792 2024-03-04T17:50:48.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.189+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=BQ6GX7QBLPA1 type=del 2024-03-04T17:50:48.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.193+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~5G7TJ326V5U5.0 size=544114 2024-03-04T17:50:48.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.193+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=5G7TJ326V5U5 tag=TUQF5I6RCO0W type=write size=1592690 2024-03-04T17:50:48.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.197+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~3YTL0SYO3S39.0 size=1048576 2024-03-04T17:50:48.199 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.201+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~3YTL0SYO3S39.1 size=1048576 2024-03-04T17:50:48.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.205+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~3YTL0SYO3S39.2 size=797022 2024-03-04T17:50:48.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.209+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=3YTL0SYO3S39 tag=E4YTNUJZW539 type=write size=3942750 2024-03-04T17:50:48.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.213+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=LEVSUHI1INVA type=del size=2700465 2024-03-04T17:50:48.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.217+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=704M7SLWXP40 type=del 2024-03-04T17:50:48.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.221+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=GC38HXTAGVRI type=del size=978790 2024-03-04T17:50:48.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.225+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=19EC44I8443Y type=write size=359593 2024-03-04T17:50:48.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.229+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=0WRYEE8LWD9M type=del size=297014 2024-03-04T17:50:48.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.233+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=79FYHVN80SFI type=del 2024-03-04T17:50:48.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.237+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=LA0R0NAQZ0DE type=del size=2564414 2024-03-04T17:50:48.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.241+0000 7fda08af98c0 1 < completed write operation key=3UWSL7NJLACY tag=LRDTES2F0QQN type=write size=251448 2024-03-04T17:50:48.245 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.245+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~VH3AC76IENJT.0 size=1048576 2024-03-04T17:50:48.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.249+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~VH3AC76IENJT.1 size=91633 2024-03-04T17:50:48.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.249+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=VH3AC76IENJT tag=AUD7MBBC31A1 type=write size=2188785 2024-03-04T17:50:48.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.253+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=4RWG4JLG4GSE type=del 2024-03-04T17:50:48.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.261+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~3K34OUVYE87G.0 size=1048576 2024-03-04T17:50:48.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.261+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~3K34OUVYE87G.1 size=384421 2024-03-04T17:50:48.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.261+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=3K34OUVYE87G tag=HNKZG09A7W1E type=write size=2481573 2024-03-04T17:50:48.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.265+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=THH25Y7XGS5L type=del 2024-03-04T17:50:48.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.269+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=0MCH781UAAEI type=write size=730366 2024-03-04T17:50:48.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.277+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=S0V61JJKB6MM type=del 2024-03-04T17:50:48.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.285+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~M66H0KUAKNTM.0 size=1048576 2024-03-04T17:50:48.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.289+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~M66H0KUAKNTM.1 size=301585 2024-03-04T17:50:48.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.289+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=M66H0KUAKNTM tag=P47YYXEXE4UZ type=write size=2398737 2024-03-04T17:50:48.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.293+0000 7fda08af98c0 1 < suggested remove operation key=FMPZ2548LBBF tag=AZT5554DISEY type=del 2024-03-04T17:50:48.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.301+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~ENNG267RGAL4.0 size=1048576 2024-03-04T17:50:48.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.305+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~ENNG267RGAL4.1 size=252049 2024-03-04T17:50:48.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.305+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=ENNG267RGAL4 tag=YVWGNJZJUNEN type=write size=2349201 2024-03-04T17:50:48.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.309+0000 7fda08af98c0 1 < canceled operation key=BYQFZSJ3WH07 tag=QFGV7Y13KBQ8 type=del size=1229364 2024-03-04T17:50:48.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.313+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=JTQ8IP2MEZAR type=del size=422016 2024-03-04T17:50:48.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.317+0000 7fda08af98c0 1 < completed write operation key=BYQFZSJ3WH07 tag=3XSYXJK0UPTK type=write size=17882 2024-03-04T17:50:48.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.321+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=38I0QIYIL1ER type=del size=384170 2024-03-04T17:50:48.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.325+0000 7fda08af98c0 -1 < failed to complete operation key=M5BMX7B7RZYL tag=UNGGFD2DC1AM type=del size=1488638: (22) Invalid argument 2024-03-04T17:50:48.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.329+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=LYY2SIVNS2WR type=del size=1302473 2024-03-04T17:50:48.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.333+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~CT44ZKOA71SW.0 size=1048576 2024-03-04T17:50:48.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.337+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~CT44ZKOA71SW.1 size=944700 2024-03-04T17:50:48.340 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.341+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=CT44ZKOA71SW tag=R1414B978WXX type=write size=3041852 2024-03-04T17:50:48.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.345+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=6R7FS1Y131FR type=write size=913519 2024-03-04T17:50:48.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.349+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~H4C0U4CH6XHL.0 size=840424 2024-03-04T17:50:48.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.357+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=H4C0U4CH6XHL tag=VAAILOW2YD5Q type=write size=1889000 2024-03-04T17:50:48.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.361+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=502AV13T7U28 type=del size=1706372 2024-03-04T17:50:48.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.369+0000 7fda08af98c0 1 < suggested remove operation key=IJSYAVR5SWPS tag=G8M379GFYZEE type=del 2024-03-04T17:50:48.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.373+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=2J5E67C1WVX7 type=del size=576826 2024-03-04T17:50:48.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.377+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~5G7TJ326V5U5.0 size=544114 2024-03-04T17:50:48.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.385+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=5G7TJ326V5U5 tag=TUQF5I6RCO0W type=write size=1592690 2024-03-04T17:50:48.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.389+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=NV2KF9QKC15M type=del size=375258 2024-03-04T17:50:48.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.393+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=LEVSUHI1INVA type=del 2024-03-04T17:50:48.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.393+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=PCV8ZHJD3DJT type=del size=1517311 2024-03-04T17:50:48.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.401+0000 7fda08af98c0 1 < suggested update operation key=3UWSL7NJLACY tag=GC38HXTAGVRI type=del size=978790 2024-03-04T17:50:48.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.405+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=1UM715A5OY3D type=del size=970792 2024-03-04T17:50:48.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.409+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=0WRYEE8LWD9M type=del 2024-03-04T17:50:48.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.417+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~MKRNL0HQ3W12.0 size=1048576 2024-03-04T17:50:48.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.421+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~MKRNL0HQ3W12.1 size=1048576 2024-03-04T17:50:48.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.425+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~MKRNL0HQ3W12.2 size=819293 2024-03-04T17:50:48.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.425+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=MKRNL0HQ3W12 tag=PC3FNA0VJ3NE type=write size=3965021 2024-03-04T17:50:48.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.433+0000 7fda08af98c0 1 < suggested remove operation key=XG5I3SDRE5JM tag=LA0R0NAQZ0DE type=del 2024-03-04T17:50:48.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.437+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=R2P75OGG9NWO type=del size=204255 2024-03-04T17:50:48.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.441+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~VH3AC76IENJT.0 size=1048576 2024-03-04T17:50:48.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.445+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~VH3AC76IENJT.1 size=91633 2024-03-04T17:50:48.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.449+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=VH3AC76IENJT tag=AUD7MBBC31A1 type=write size=2188785 2024-03-04T17:50:48.455 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.457+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~CSXJSGXGD6MP.0 size=912017 2024-03-04T17:50:48.455 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.457+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=CSXJSGXGD6MP tag=XW02VCY35OPW type=write size=1960593 2024-03-04T17:50:48.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.461+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~3K34OUVYE87G.0 size=1048576 2024-03-04T17:50:48.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.465+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~3K34OUVYE87G.1 size=384421 2024-03-04T17:50:48.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.469+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=3K34OUVYE87G tag=HNKZG09A7W1E type=write size=2481573 2024-03-04T17:50:48.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.477+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~CJA1KSIOLRCD.0 size=1048576 2024-03-04T17:50:48.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.481+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~CJA1KSIOLRCD.1 size=641164 2024-03-04T17:50:48.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.481+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=CJA1KSIOLRCD tag=L47U42LD3IL5 type=write size=2738316 2024-03-04T17:50:48.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.489+0000 7fda08af98c0 1 < suggested remove operation key=1OZKPHMPV3OQ tag=0MCH781UAAEI type=write 2024-03-04T17:50:48.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.493+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=F6H60O322TWY type=del size=2946027 2024-03-04T17:50:48.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.497+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~M66H0KUAKNTM.0 size=1048576 2024-03-04T17:50:48.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.497+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~M66H0KUAKNTM.1 size=301585 2024-03-04T17:50:48.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.505+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=M66H0KUAKNTM tag=P47YYXEXE4UZ type=write size=2398737 2024-03-04T17:50:48.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.513+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~JW2RJ35F5W1Y.0 size=1048576 2024-03-04T17:50:48.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.517+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~JW2RJ35F5W1Y.1 size=330611 2024-03-04T17:50:48.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.517+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=JW2RJ35F5W1Y tag=O87QW49XWTUK type=write size=2427763 2024-03-04T17:50:48.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.521+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~ENNG267RGAL4.0 size=1048576 2024-03-04T17:50:48.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.521+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~ENNG267RGAL4.1 size=252049 2024-03-04T17:50:48.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.529+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=ENNG267RGAL4 tag=YVWGNJZJUNEN type=write size=2349201 2024-03-04T17:50:48.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.533+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=TTDSLTU3S3SU type=write size=1044159 2024-03-04T17:50:48.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.537+0000 7fda08af98c0 -1 < failed to complete operation key=IJSYAVR5SWPS tag=JTQ8IP2MEZAR type=del size=422016: (22) Invalid argument 2024-03-04T17:50:48.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.541+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=6UC9Y8B6OTX0 type=del size=1769735 2024-03-04T17:50:48.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.545+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=38I0QIYIL1ER type=del 2024-03-04T17:50:48.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.549+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=OEBW5YIUSMJY type=del size=1353340 2024-03-04T17:50:48.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.553+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=LYY2SIVNS2WR type=del 2024-03-04T17:50:48.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.557+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~BM7FL9MFUQ2P.0 size=540618 2024-03-04T17:50:48.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.557+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=BM7FL9MFUQ2P tag=CJO67IXF31XP type=write size=1589194 2024-03-04T17:50:48.561 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.561+0000 7fda08af98c0 1 < completed write operation key=09KBCA2XURTF tag=6R7FS1Y131FR type=write size=913519 2024-03-04T17:50:48.565 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.565+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=5ZWNWQPBRBSE type=del size=3303361 2024-03-04T17:50:48.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.569+0000 7fda08af98c0 1 < canceled operation key=GDGTJZ21TG32 tag=502AV13T7U28 type=del size=1706372 2024-03-04T17:50:48.573 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.573+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=2U8MBJAQ28GM type=del size=2099005 2024-03-04T17:50:48.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.577+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=2J5E67C1WVX7 type=del 2024-03-04T17:50:48.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.581+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=3H3384970ZWB type=del size=3016838 2024-03-04T17:50:48.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.589+0000 7fda08af98c0 1 < suggested update operation key=TOFWDFH0HL3Q tag=NV2KF9QKC15M type=del size=375258 2024-03-04T17:50:48.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.593+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=L0FOM9WY27ZK type=del size=3682572 2024-03-04T17:50:48.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.597+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=PCV8ZHJD3DJT type=del 2024-03-04T17:50:48.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.601+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=5355ARE012S9 type=del size=186165 2024-03-04T17:50:48.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.605+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=1UM715A5OY3D type=del 2024-03-04T17:50:48.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.609+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=G0G5PJ4HL6QT type=write size=145707 2024-03-04T17:50:48.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.613+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~MKRNL0HQ3W12.0 size=1048576 2024-03-04T17:50:48.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.617+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~MKRNL0HQ3W12.1 size=1048576 2024-03-04T17:50:48.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.617+0000 7fda08af98c0 1 < canceled part key=_multipart_3UWSL7NJLACY.2~MKRNL0HQ3W12.2 size=819293 2024-03-04T17:50:48.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.625+0000 7fda08af98c0 1 < canceled multipart upload key=3UWSL7NJLACY upload=MKRNL0HQ3W12 tag=PC3FNA0VJ3NE type=write size=3965021 2024-03-04T17:50:48.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.633+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~717HYKZZ8PTW.0 size=319728 2024-03-04T17:50:48.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.633+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=717HYKZZ8PTW tag=PQ0H0RQIUP7Z type=write size=1368304 2024-03-04T17:50:48.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.641+0000 7fda08af98c0 1 < suggested remove operation key=VJVORUG95FLC tag=R2P75OGG9NWO type=del 2024-03-04T17:50:48.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.649+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=INEIG1SYCPLA type=write size=903608 2024-03-04T17:50:48.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.653+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~CSXJSGXGD6MP.0 size=912017 2024-03-04T17:50:48.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.657+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=CSXJSGXGD6MP tag=XW02VCY35OPW type=write size=1960593 2024-03-04T17:50:48.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.661+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=S749CJV9RRPC type=del size=356178 2024-03-04T17:50:48.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.665+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~CJA1KSIOLRCD.0 size=1048576 2024-03-04T17:50:48.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.669+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~CJA1KSIOLRCD.1 size=641164 2024-03-04T17:50:48.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.677+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=CJA1KSIOLRCD tag=L47U42LD3IL5 type=write size=2738316 2024-03-04T17:50:48.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.681+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~S78QF2MCDSRO.0 size=1048576 2024-03-04T17:50:48.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.685+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~S78QF2MCDSRO.1 size=1048576 2024-03-04T17:50:48.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.689+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~S78QF2MCDSRO.2 size=65375 2024-03-04T17:50:48.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.689+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=S78QF2MCDSRO tag=2T3XCQN9MLQZ type=write size=3211103 2024-03-04T17:50:48.690 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.693+0000 7fda08af98c0 -1 < failed to complete operation key=TOFWDFH0HL3Q tag=F6H60O322TWY type=del size=2946027: (22) Invalid argument 2024-03-04T17:50:48.697 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.701+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~9EB1RVP2F5RT.0 size=1048576 2024-03-04T17:50:48.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.701+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~9EB1RVP2F5RT.1 size=606963 2024-03-04T17:50:48.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.701+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=9EB1RVP2F5RT tag=28VDYYCW5BWU type=write size=2704115 2024-03-04T17:50:48.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.705+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~JW2RJ35F5W1Y.0 size=1048576 2024-03-04T17:50:48.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.709+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~JW2RJ35F5W1Y.1 size=330611 2024-03-04T17:50:48.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.717+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=JW2RJ35F5W1Y tag=O87QW49XWTUK type=write size=2427763 2024-03-04T17:50:48.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.721+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=VUJY29VLS33X type=del size=3907381 2024-03-04T17:50:48.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.725+0000 7fda08af98c0 1 < completed write operation key=1OZKPHMPV3OQ tag=TTDSLTU3S3SU type=write size=1044159 2024-03-04T17:50:48.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.729+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=993QLINZAJ4Q type=write size=741784 2024-03-04T17:50:48.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.733+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=6UC9Y8B6OTX0 type=del 2024-03-04T17:50:48.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.737+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=38ZT4P3K0AWS type=del size=881993 2024-03-04T17:50:48.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.741+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=OEBW5YIUSMJY type=del 2024-03-04T17:50:48.745 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.749+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~1HJ7IZMY00BP.0 size=720431 2024-03-04T17:50:48.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.749+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=1HJ7IZMY00BP tag=2RHTJT3ML4PS type=write size=1769007 2024-03-04T17:50:48.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.753+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~BM7FL9MFUQ2P.0 size=540618 2024-03-04T17:50:48.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.757+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=BM7FL9MFUQ2P tag=CJO67IXF31XP type=write size=1589194 2024-03-04T17:50:48.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.765+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~ZRPCI6U7RXVA.0 size=1048576 2024-03-04T17:50:48.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.769+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~ZRPCI6U7RXVA.1 size=741375 2024-03-04T17:50:48.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.769+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=ZRPCI6U7RXVA tag=DG2OXQ0001PX type=write size=2838527 2024-03-04T17:50:48.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.769+0000 7fda08af98c0 1 < canceled operation key=8YL7C9PI58TK tag=5ZWNWQPBRBSE type=del size=3303361 2024-03-04T17:50:48.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.777+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~G2DV0CW1AMZF.0 size=1048576 2024-03-04T17:50:48.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.781+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~G2DV0CW1AMZF.1 size=1048576 2024-03-04T17:50:48.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.781+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~G2DV0CW1AMZF.2 size=526304 2024-03-04T17:50:48.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.781+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=G2DV0CW1AMZF tag=QJVA1YRJGU90 type=write size=3672032 2024-03-04T17:50:48.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.785+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=2U8MBJAQ28GM type=del 2024-03-04T17:50:48.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.793+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~WJUZTJ3DJUEA.0 size=1048576 2024-03-04T17:50:48.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.797+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~WJUZTJ3DJUEA.1 size=1048576 2024-03-04T17:50:48.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.797+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~WJUZTJ3DJUEA.2 size=986095 2024-03-04T17:50:48.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.797+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=WJUZTJ3DJUEA tag=PCZQ0WAJQ8RY type=write size=4131823 2024-03-04T17:50:48.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.805+0000 7fda08af98c0 1 < suggested remove operation key=ITIVF3UAT9NK tag=3H3384970ZWB type=del 2024-03-04T17:50:48.812 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.813+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~HNEFUAUR7I17.0 size=1048576 2024-03-04T17:50:48.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.817+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~HNEFUAUR7I17.1 size=308847 2024-03-04T17:50:48.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.817+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=HNEFUAUR7I17 tag=I2YQLBGDOVQC type=write size=2405999 2024-03-04T17:50:48.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.825+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=L0FOM9WY27ZK type=del 2024-03-04T17:50:48.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.829+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=5U7NKJSKK411 type=del size=1593479 2024-03-04T17:50:48.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.833+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=5355ARE012S9 type=del 2024-03-04T17:50:48.841 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.841+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~491GB76O8EGE.0 size=1048576 2024-03-04T17:50:48.844 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.845+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~491GB76O8EGE.1 size=1048576 2024-03-04T17:50:48.848 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.849+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~491GB76O8EGE.2 size=782636 2024-03-04T17:50:48.848 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.849+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=491GB76O8EGE tag=WOV70IJN9ENR type=write size=3928364 2024-03-04T17:50:48.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.853+0000 7fda08af98c0 1 < completed write operation key=8YL7C9PI58TK tag=G0G5PJ4HL6QT type=write size=145707 2024-03-04T17:50:48.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.857+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=9CLI540X3TIT type=del size=430669 2024-03-04T17:50:48.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.865+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~717HYKZZ8PTW.0 size=319728 2024-03-04T17:50:48.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.869+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=717HYKZZ8PTW tag=PQ0H0RQIUP7Z type=write size=1368304 2024-03-04T17:50:48.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.873+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=IXR3RNPVINYW type=write size=143737 2024-03-04T17:50:48.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.877+0000 7fda08af98c0 1 < completed write operation key=XG5I3SDRE5JM tag=INEIG1SYCPLA type=write size=903608 2024-03-04T17:50:48.885 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.885+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~0MADLMDK8724.0 size=1048576 2024-03-04T17:50:48.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.889+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~0MADLMDK8724.1 size=268537 2024-03-04T17:50:48.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.889+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=0MADLMDK8724 tag=5CLEKMVI9C7E type=write size=2365689 2024-03-04T17:50:48.893 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.893+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=S749CJV9RRPC type=del size=356178 2024-03-04T17:50:48.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.901+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=T1UA4PV330P0 type=del size=3179319 2024-03-04T17:50:48.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.905+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~S78QF2MCDSRO.0 size=1048576 2024-03-04T17:50:48.905 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.905+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~S78QF2MCDSRO.1 size=1048576 2024-03-04T17:50:48.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.909+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~S78QF2MCDSRO.2 size=65375 2024-03-04T17:50:48.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.917+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=S78QF2MCDSRO tag=2T3XCQN9MLQZ type=write size=3211103 2024-03-04T17:50:48.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.921+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=ZF0TFM0OJG4L type=write size=140424 2024-03-04T17:50:48.921 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.925+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~9EB1RVP2F5RT.0 size=1048576 2024-03-04T17:50:48.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.925+0000 7fda08af98c0 1 < canceled part key=_multipart_BYQFZSJ3WH07.2~9EB1RVP2F5RT.1 size=606963 2024-03-04T17:50:48.927 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.929+0000 7fda08af98c0 1 < canceled multipart upload key=BYQFZSJ3WH07 upload=9EB1RVP2F5RT tag=28VDYYCW5BWU type=write size=2704115 2024-03-04T17:50:48.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.933+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=LVLTOB10BO0Y type=del size=3078360 2024-03-04T17:50:48.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.941+0000 7fda08af98c0 1 < suggested remove operation key=3UWSL7NJLACY tag=VUJY29VLS33X type=del 2024-03-04T17:50:48.944 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.945+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=5YOHAKJHBWZR type=del size=762349 2024-03-04T17:50:48.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.949+0000 7fda08af98c0 1 < completed write operation key=TOFWDFH0HL3Q tag=993QLINZAJ4Q type=write size=741784 2024-03-04T17:50:48.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.953+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=H9PJLVH0PAYW type=del size=479316 2024-03-04T17:50:48.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.957+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=38ZT4P3K0AWS type=del 2024-03-04T17:50:48.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.961+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=LA27QBZA7IGH type=del size=1237242 2024-03-04T17:50:48.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.965+0000 7fda08af98c0 1 < canceled part key=_multipart_3UWSL7NJLACY.2~1HJ7IZMY00BP.0 size=720431 2024-03-04T17:50:48.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.969+0000 7fda08af98c0 1 < canceled multipart upload key=3UWSL7NJLACY upload=1HJ7IZMY00BP tag=2RHTJT3ML4PS type=write size=1769007 2024-03-04T17:50:48.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.973+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=HPD1I3WZ6Z2I type=del size=998922 2024-03-04T17:50:48.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.977+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~ZRPCI6U7RXVA.0 size=1048576 2024-03-04T17:50:48.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.981+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~ZRPCI6U7RXVA.1 size=741375 2024-03-04T17:50:48.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.985+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=ZRPCI6U7RXVA tag=DG2OXQ0001PX type=write size=2838527 2024-03-04T17:50:48.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.993+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~0GNXDEUIN2KV.0 size=1048576 2024-03-04T17:50:48.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.993+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~0GNXDEUIN2KV.1 size=1048576 2024-03-04T17:50:48.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.997+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~0GNXDEUIN2KV.2 size=722912 2024-03-04T17:50:48.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:48.997+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=0GNXDEUIN2KV tag=8RMFIEV7221V type=write size=3868640 2024-03-04T17:50:49.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.001+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~G2DV0CW1AMZF.0 size=1048576 2024-03-04T17:50:49.003 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.005+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~G2DV0CW1AMZF.1 size=1048576 2024-03-04T17:50:49.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.013+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~G2DV0CW1AMZF.2 size=526304 2024-03-04T17:50:49.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.017+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=G2DV0CW1AMZF tag=QJVA1YRJGU90 type=write size=3672032 2024-03-04T17:50:49.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.025+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~MCFWKD06MROE.0 size=284890 2024-03-04T17:50:49.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.025+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=MCFWKD06MROE tag=JY213T7DUB5H type=write size=1333466 2024-03-04T17:50:49.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.029+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~WJUZTJ3DJUEA.0 size=1048576 2024-03-04T17:50:49.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.029+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~WJUZTJ3DJUEA.1 size=1048576 2024-03-04T17:50:49.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.033+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~WJUZTJ3DJUEA.2 size=986095 2024-03-04T17:50:49.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.041+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=WJUZTJ3DJUEA tag=PCZQ0WAJQ8RY type=write size=4131823 2024-03-04T17:50:49.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.045+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~FH2WDLI6Y6ZD.0 size=392217 2024-03-04T17:50:49.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.045+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=FH2WDLI6Y6ZD tag=0J932E5PTAJU type=write size=1440793 2024-03-04T17:50:49.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.053+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~HNEFUAUR7I17.0 size=1048576 2024-03-04T17:50:49.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.053+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~HNEFUAUR7I17.1 size=308847 2024-03-04T17:50:49.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.061+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=HNEFUAUR7I17 tag=I2YQLBGDOVQC type=write size=2405999 2024-03-04T17:50:49.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.065+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=EAHP6AI3FZGU type=write size=3904 2024-03-04T17:50:49.067 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.069+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=5U7NKJSKK411 type=del 2024-03-04T17:50:49.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.073+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=J9Q4XO4L6XA5 type=del size=1819211 2024-03-04T17:50:49.076 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.077+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~491GB76O8EGE.0 size=1048576 2024-03-04T17:50:49.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.081+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~491GB76O8EGE.1 size=1048576 2024-03-04T17:50:49.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.085+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~491GB76O8EGE.2 size=782636 2024-03-04T17:50:49.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.089+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=491GB76O8EGE tag=WOV70IJN9ENR type=write size=3928364 2024-03-04T17:50:49.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.093+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=4E79Q2DXKRWQ type=del size=1977858 2024-03-04T17:50:49.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.097+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=9CLI540X3TIT type=del 2024-03-04T17:50:49.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.101+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=YZK7MEGIBLHN type=del size=211134 2024-03-04T17:50:49.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.109+0000 7fda08af98c0 1 < completed write operation key=750NSVFBS03A tag=IXR3RNPVINYW type=write size=143737 2024-03-04T17:50:49.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.113+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=Q2VR3E3XUHRZ type=del size=3856602 2024-03-04T17:50:49.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.117+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~0MADLMDK8724.0 size=1048576 2024-03-04T17:50:49.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.117+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~0MADLMDK8724.1 size=268537 2024-03-04T17:50:49.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.125+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=0MADLMDK8724 tag=5CLEKMVI9C7E type=write size=2365689 2024-03-04T17:50:49.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.133+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~D0Z3Y9ZQDJII.0 size=1048576 2024-03-04T17:50:49.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.133+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~D0Z3Y9ZQDJII.1 size=619738 2024-03-04T17:50:49.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.133+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=D0Z3Y9ZQDJII tag=IYISK8O1MQZ7 type=write size=2716890 2024-03-04T17:50:49.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.141+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=T1UA4PV330P0 type=del 2024-03-04T17:50:49.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.145+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~D64ADBNZ0FJ3.0 size=1048576 2024-03-04T17:50:49.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.149+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~D64ADBNZ0FJ3.1 size=216661 2024-03-04T17:50:49.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.149+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=D64ADBNZ0FJ3 tag=6B8XB088EFTP type=write size=2313813 2024-03-04T17:50:49.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.153+0000 7fda08af98c0 -1 < failed to complete operation key=3UWSL7NJLACY tag=ZF0TFM0OJG4L type=write size=140424: (22) Invalid argument 2024-03-04T17:50:49.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.161+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~03Z1Y3FLV6ZV.0 size=1048576 2024-03-04T17:50:49.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.165+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~03Z1Y3FLV6ZV.1 size=354204 2024-03-04T17:50:49.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.165+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=03Z1Y3FLV6ZV tag=BKSZ15TNMTHC type=write size=2451356 2024-03-04T17:50:49.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.169+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=LVLTOB10BO0Y type=del 2024-03-04T17:50:49.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.177+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~IPQFTTSZHTXT.0 size=1048576 2024-03-04T17:50:49.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.177+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~IPQFTTSZHTXT.1 size=1048576 2024-03-04T17:50:49.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.181+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~IPQFTTSZHTXT.2 size=923238 2024-03-04T17:50:49.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.181+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=IPQFTTSZHTXT tag=CU9QNAX2QU1O type=write size=4068966 2024-03-04T17:50:49.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.185+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=5YOHAKJHBWZR type=del 2024-03-04T17:50:49.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.189+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=JSCIJOS5SVWB type=del size=146025 2024-03-04T17:50:49.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.193+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=H9PJLVH0PAYW type=del 2024-03-04T17:50:49.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.197+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=67736X8JM248 type=del size=1156922 2024-03-04T17:50:49.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.201+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=LA27QBZA7IGH type=del 2024-03-04T17:50:49.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.205+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=B43OVUODN44E type=write size=349841 2024-03-04T17:50:49.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.213+0000 7fda08af98c0 1 < suggested update operation key=XG2XYBCMUNY5 tag=HPD1I3WZ6Z2I type=del size=998922 2024-03-04T17:50:49.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.217+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=CEUNH7AMIESM type=del size=4124234 2024-03-04T17:50:49.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.221+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~0GNXDEUIN2KV.0 size=1048576 2024-03-04T17:50:49.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.225+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~0GNXDEUIN2KV.1 size=1048576 2024-03-04T17:50:49.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.229+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~0GNXDEUIN2KV.2 size=722912 2024-03-04T17:50:49.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.233+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=0GNXDEUIN2KV tag=8RMFIEV7221V type=write size=3868640 2024-03-04T17:50:49.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.237+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=K0VAN2R6F38Y type=del size=3655837 2024-03-04T17:50:49.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.241+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~MCFWKD06MROE.0 size=284890 2024-03-04T17:50:49.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.249+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=MCFWKD06MROE tag=JY213T7DUB5H type=write size=1333466 2024-03-04T17:50:49.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.253+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=PZ5Y2L4JXPOZ type=del size=2638212 2024-03-04T17:50:49.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.257+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~FH2WDLI6Y6ZD.0 size=392217 2024-03-04T17:50:49.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.261+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=FH2WDLI6Y6ZD tag=0J932E5PTAJU type=write size=1440793 2024-03-04T17:50:49.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.269+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=X00EL7QTABHK type=del size=656696 2024-03-04T17:50:49.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.273+0000 7fda08af98c0 1 < completed write operation key=DJCCQ6S0E7UG tag=EAHP6AI3FZGU type=write size=3904 2024-03-04T17:50:49.277 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.277+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~291FUIXWJSV2.0 size=1048576 2024-03-04T17:50:49.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.281+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~291FUIXWJSV2.1 size=1048576 2024-03-04T17:50:49.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.285+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~291FUIXWJSV2.2 size=171877 2024-03-04T17:50:49.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.285+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=291FUIXWJSV2 tag=ZB1VJF18SPBR type=write size=3317605 2024-03-04T17:50:49.287 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.289+0000 7fda08af98c0 1 < canceled operation key=G0BHMSCO0XLF tag=J9Q4XO4L6XA5 type=del size=1819211 2024-03-04T17:50:49.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.293+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=91J6JY607QOD type=del size=2164637 2024-03-04T17:50:49.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.297+0000 7fda08af98c0 -1 < failed to complete operation key=XG2XYBCMUNY5 tag=4E79Q2DXKRWQ type=del size=1977858: (22) Invalid argument 2024-03-04T17:50:49.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.301+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=YFM5NK4IWPA4 type=del size=3070374 2024-03-04T17:50:49.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.309+0000 7fda08af98c0 1 < suggested remove operation key=BYQFZSJ3WH07 tag=YZK7MEGIBLHN type=del 2024-03-04T17:50:49.310 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.313+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=T5BV17OC7ENT type=del size=3380278 2024-03-04T17:50:49.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.317+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=Q2VR3E3XUHRZ type=del 2024-03-04T17:50:49.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.325+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~VZFZWBWE092S.0 size=1048576 2024-03-04T17:50:49.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.325+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~VZFZWBWE092S.1 size=1048576 2024-03-04T17:50:49.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.329+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~VZFZWBWE092S.2 size=17532 2024-03-04T17:50:49.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.329+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=VZFZWBWE092S tag=EPOZ01OG20V6 type=write size=3163260 2024-03-04T17:50:49.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.333+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~D0Z3Y9ZQDJII.0 size=1048576 2024-03-04T17:50:49.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.337+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~D0Z3Y9ZQDJII.1 size=619738 2024-03-04T17:50:49.340 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.341+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=D0Z3Y9ZQDJII tag=IYISK8O1MQZ7 type=write size=2716890 2024-03-04T17:50:49.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.345+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=OTWYLN60K0JM type=del size=1958106 2024-03-04T17:50:49.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.345+0000 7fda08af98c0 1 < canceled part key=_multipart_IJSYAVR5SWPS.2~D64ADBNZ0FJ3.0 size=1048576 2024-03-04T17:50:49.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.349+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~D64ADBNZ0FJ3.1 size=216661 2024-03-04T17:50:49.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.357+0000 7fda08af98c0 1 < canceled multipart upload key=IJSYAVR5SWPS upload=D64ADBNZ0FJ3 tag=6B8XB088EFTP type=write size=2313813 2024-03-04T17:50:49.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.361+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=SENY8LUFBLFW type=write size=77392 2024-03-04T17:50:49.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.365+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~03Z1Y3FLV6ZV.0 size=1048576 2024-03-04T17:50:49.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.365+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~03Z1Y3FLV6ZV.1 size=354204 2024-03-04T17:50:49.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.377+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=03Z1Y3FLV6ZV tag=BKSZ15TNMTHC type=write size=2451356 2024-03-04T17:50:49.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.385+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~N2942R33V2R3.0 size=343069 2024-03-04T17:50:49.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.385+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=N2942R33V2R3 tag=J5FD6N332R12 type=write size=1391645 2024-03-04T17:50:49.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.389+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~IPQFTTSZHTXT.0 size=1048576 2024-03-04T17:50:49.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.389+0000 7fda08af98c0 1 < canceled part key=_multipart_BYQFZSJ3WH07.2~IPQFTTSZHTXT.1 size=1048576 2024-03-04T17:50:49.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.389+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~IPQFTTSZHTXT.2 size=923238 2024-03-04T17:50:49.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.397+0000 7fda08af98c0 1 < canceled multipart upload key=BYQFZSJ3WH07 upload=IPQFTTSZHTXT tag=CU9QNAX2QU1O type=write size=4068966 2024-03-04T17:50:49.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.405+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~VEEQ577MK11Q.0 size=46248 2024-03-04T17:50:49.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.405+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=VEEQ577MK11Q tag=FY0XIZWHTEOQ type=write size=1094824 2024-03-04T17:50:49.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.409+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=JSCIJOS5SVWB type=del 2024-03-04T17:50:49.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.413+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=BXZKNC6RYQOX type=del size=3877429 2024-03-04T17:50:49.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.417+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=67736X8JM248 type=del 2024-03-04T17:50:49.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.425+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~JN6YX6VTTTG1.0 size=386127 2024-03-04T17:50:49.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.425+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=JN6YX6VTTTG1 tag=9NNGUOMI6WV9 type=write size=1434703 2024-03-04T17:50:49.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.429+0000 7fda08af98c0 -1 < failed to complete operation key=XG2XYBCMUNY5 tag=B43OVUODN44E type=write size=349841: (22) Invalid argument 2024-03-04T17:50:49.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.429+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=TXFQTEGWCZJ5 type=del size=1524583 2024-03-04T17:50:49.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.441+0000 7fda08af98c0 1 < suggested remove operation key=750NSVFBS03A tag=CEUNH7AMIESM type=del 2024-03-04T17:50:49.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.441+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=6O5IUIVU9ZTK type=del size=2682749 2024-03-04T17:50:49.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.445+0000 7fda08af98c0 1 < canceled operation key=C4B45GL40IMZ tag=K0VAN2R6F38Y type=del size=3655837 2024-03-04T17:50:49.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.449+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=7OJ59WYVS6M3 type=del size=3392371 2024-03-04T17:50:49.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.453+0000 7fda08af98c0 1 < canceled operation key=DJCCQ6S0E7UG tag=PZ5Y2L4JXPOZ type=del size=2638212 2024-03-04T17:50:49.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.457+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=TAK07VOF4JVW type=del size=515564 2024-03-04T17:50:49.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.461+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=X00EL7QTABHK type=del 2024-03-04T17:50:49.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.465+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=SMS1Y08CC2YG type=del size=1589237 2024-03-04T17:50:49.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.469+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~291FUIXWJSV2.0 size=1048576 2024-03-04T17:50:49.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.473+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~291FUIXWJSV2.1 size=1048576 2024-03-04T17:50:49.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.473+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~291FUIXWJSV2.2 size=171877 2024-03-04T17:50:49.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.481+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=291FUIXWJSV2 tag=ZB1VJF18SPBR type=write size=3317605 2024-03-04T17:50:49.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.485+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~2C2WAUW07ZIJ.0 size=1048576 2024-03-04T17:50:49.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.489+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~2C2WAUW07ZIJ.1 size=1048576 2024-03-04T17:50:49.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.493+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~2C2WAUW07ZIJ.2 size=611325 2024-03-04T17:50:49.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.493+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=2C2WAUW07ZIJ tag=K85FPFJ3GN30 type=write size=3757053 2024-03-04T17:50:49.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.497+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=91J6JY607QOD type=del 2024-03-04T17:50:49.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.505+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~0KL9HW3UMHKO.0 size=330449 2024-03-04T17:50:49.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.505+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=0KL9HW3UMHKO tag=LEY4Q8N6EA86 type=write size=1379025 2024-03-04T17:50:49.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.509+0000 7fda08af98c0 1 < canceled operation key=DJCCQ6S0E7UG tag=YFM5NK4IWPA4 type=del size=3070374 2024-03-04T17:50:49.509 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.509+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=YTGPVZX3LYWN type=del size=1605320 2024-03-04T17:50:49.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.517+0000 7fda08af98c0 1 < suggested update operation key=HCA7QLGM2O4O tag=T5BV17OC7ENT type=del size=3380278 2024-03-04T17:50:49.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.525+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~HML3C6GS9U51.0 size=1048576 2024-03-04T17:50:49.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.529+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~HML3C6GS9U51.1 size=1048576 2024-03-04T17:50:49.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.533+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~HML3C6GS9U51.2 size=149854 2024-03-04T17:50:49.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.533+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=HML3C6GS9U51 tag=P27LOV016DVN type=write size=3295582 2024-03-04T17:50:49.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.537+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~VZFZWBWE092S.0 size=1048576 2024-03-04T17:50:49.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.537+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~VZFZWBWE092S.1 size=1048576 2024-03-04T17:50:49.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.541+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~VZFZWBWE092S.2 size=17532 2024-03-04T17:50:49.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.549+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=VZFZWBWE092S tag=EPOZ01OG20V6 type=write size=3163260 2024-03-04T17:50:49.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.549+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=BSOQZ1PTW52V type=del size=2024256 2024-03-04T17:50:49.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.553+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=OTWYLN60K0JM type=del 2024-03-04T17:50:49.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.557+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=9YQUNDXS3408 type=del size=87737 2024-03-04T17:50:49.561 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.561+0000 7fda08af98c0 1 < completed write operation key=AO3QS6CM7I5D tag=SENY8LUFBLFW type=write size=77392 2024-03-04T17:50:49.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.565+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=AAZU2KVL5I11 type=del size=2043155 2024-03-04T17:50:49.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.569+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~N2942R33V2R3.0 size=343069 2024-03-04T17:50:49.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.577+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=N2942R33V2R3 tag=J5FD6N332R12 type=write size=1391645 2024-03-04T17:50:49.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.581+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~MAM5NRGGO3RJ.0 size=1048576 2024-03-04T17:50:49.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.585+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~MAM5NRGGO3RJ.1 size=152910 2024-03-04T17:50:49.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.585+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=MAM5NRGGO3RJ tag=6J2KJR7JC2H1 type=write size=2250062 2024-03-04T17:50:49.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.589+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~VEEQ577MK11Q.0 size=46248 2024-03-04T17:50:49.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.597+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=VEEQ577MK11Q tag=FY0XIZWHTEOQ type=write size=1094824 2024-03-04T17:50:49.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.601+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=5NRTMULFOBLK type=del size=2793870 2024-03-04T17:50:49.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.605+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=BXZKNC6RYQOX type=del 2024-03-04T17:50:49.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.609+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=72YQLNL1OOBX type=write size=725647 2024-03-04T17:50:49.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.609+0000 7fda08af98c0 1 < canceled part key=_multipart_XG2XYBCMUNY5.2~JN6YX6VTTTG1.0 size=386127 2024-03-04T17:50:49.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.617+0000 7fda08af98c0 1 < canceled multipart upload key=XG2XYBCMUNY5 upload=JN6YX6VTTTG1 tag=9NNGUOMI6WV9 type=write size=1434703 2024-03-04T17:50:49.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.625+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~IROL7E2364KX.0 size=1048576 2024-03-04T17:50:49.626 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.629+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~IROL7E2364KX.1 size=853385 2024-03-04T17:50:49.626 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.629+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=IROL7E2364KX tag=8C9E4KKAPROB type=write size=2950537 2024-03-04T17:50:49.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.629+0000 7fda08af98c0 1 < canceled operation key=JA4JK99I0942 tag=TXFQTEGWCZJ5 type=del size=1524583 2024-03-04T17:50:49.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.633+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=E9U2C0S1G23Y type=del size=2755783 2024-03-04T17:50:49.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.641+0000 7fda08af98c0 1 < suggested update operation key=IJSYAVR5SWPS tag=6O5IUIVU9ZTK type=del size=2682749 2024-03-04T17:50:49.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.645+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~BD5P87YFUE29.0 size=1048576 2024-03-04T17:50:49.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.649+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~BD5P87YFUE29.1 size=1048576 2024-03-04T17:50:49.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.653+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~BD5P87YFUE29.2 size=811467 2024-03-04T17:50:49.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.653+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=BD5P87YFUE29 tag=XVEK3QDTIE0F type=write size=3957195 2024-03-04T17:50:49.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.657+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=7OJ59WYVS6M3 type=del 2024-03-04T17:50:49.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.665+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~56KA0VMMD08Q.0 size=620939 2024-03-04T17:50:49.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.665+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=56KA0VMMD08Q tag=UAUTFRLCX27E type=write size=1669515 2024-03-04T17:50:49.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.669+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=TAK07VOF4JVW type=del 2024-03-04T17:50:49.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.673+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=FWZEL4G2QXL8 type=del size=428790 2024-03-04T17:50:49.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.677+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=SMS1Y08CC2YG type=del 2024-03-04T17:50:49.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.681+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=FPV3ULIT4FWV type=del size=3844003 2024-03-04T17:50:49.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.685+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~2C2WAUW07ZIJ.0 size=1048576 2024-03-04T17:50:49.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.685+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~2C2WAUW07ZIJ.1 size=1048576 2024-03-04T17:50:49.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.689+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~2C2WAUW07ZIJ.2 size=611325 2024-03-04T17:50:49.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.693+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=2C2WAUW07ZIJ tag=K85FPFJ3GN30 type=write size=3757053 2024-03-04T17:50:49.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.697+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=FIR8UMCUOXMI type=write size=427044 2024-03-04T17:50:49.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.701+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~0KL9HW3UMHKO.0 size=330449 2024-03-04T17:50:49.705 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.705+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=0KL9HW3UMHKO tag=LEY4Q8N6EA86 type=write size=1379025 2024-03-04T17:50:49.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.709+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=7AFLKG5TVJ17 type=del size=552778 2024-03-04T17:50:49.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.713+0000 7fda08af98c0 -1 < failed to complete operation key=IJSYAVR5SWPS tag=YTGPVZX3LYWN type=del size=1605320: (22) Invalid argument 2024-03-04T17:50:49.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.725+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~OPSVIRG3097O.0 size=1048576 2024-03-04T17:50:49.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.725+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~OPSVIRG3097O.1 size=1048576 2024-03-04T17:50:49.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.729+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~OPSVIRG3097O.2 size=785929 2024-03-04T17:50:49.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.729+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=OPSVIRG3097O tag=I6WXK08KGZVX type=write size=3931657 2024-03-04T17:50:49.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.733+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~HML3C6GS9U51.0 size=1048576 2024-03-04T17:50:49.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.737+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~HML3C6GS9U51.1 size=1048576 2024-03-04T17:50:49.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.737+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~HML3C6GS9U51.2 size=149854 2024-03-04T17:50:49.742 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.745+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=HML3C6GS9U51 tag=P27LOV016DVN type=write size=3295582 2024-03-04T17:50:49.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.749+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=3XOXYIQC5UKX type=del size=3167194 2024-03-04T17:50:49.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.753+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=BSOQZ1PTW52V type=del 2024-03-04T17:50:49.754 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.757+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=PZKX55R0Q1MF type=del size=1511944 2024-03-04T17:50:49.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.761+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=9YQUNDXS3408 type=del 2024-03-04T17:50:49.762 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.765+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=233BDK7T771R type=del size=3015687 2024-03-04T17:50:49.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.769+0000 7fda08af98c0 -1 < failed to complete operation key=IJSYAVR5SWPS tag=AAZU2KVL5I11 type=del size=2043155: (22) Invalid argument 2024-03-04T17:50:49.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.773+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~06PQL55QX2JT.0 size=770442 2024-03-04T17:50:49.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.773+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=06PQL55QX2JT tag=T05J5VIKOIB6 type=write size=1819018 2024-03-04T17:50:49.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.777+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~MAM5NRGGO3RJ.0 size=1048576 2024-03-04T17:50:49.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.781+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~MAM5NRGGO3RJ.1 size=152910 2024-03-04T17:50:49.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.785+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=MAM5NRGGO3RJ tag=6J2KJR7JC2H1 type=write size=2250062 2024-03-04T17:50:49.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.793+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~EHOLJLBA8C62.0 size=1048576 2024-03-04T17:50:49.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.797+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~EHOLJLBA8C62.1 size=450679 2024-03-04T17:50:49.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.797+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=EHOLJLBA8C62 tag=BTW1P27FCQJL type=write size=2547831 2024-03-04T17:50:49.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.797+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=5NRTMULFOBLK type=del 2024-03-04T17:50:49.805 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.805+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~JJ1KWZ18BE6L.0 size=1048576 2024-03-04T17:50:49.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.809+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~JJ1KWZ18BE6L.1 size=1048576 2024-03-04T17:50:49.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.813+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~JJ1KWZ18BE6L.2 size=653968 2024-03-04T17:50:49.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.813+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=JJ1KWZ18BE6L tag=HG4CLFG14FBI type=write size=3799696 2024-03-04T17:50:49.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.817+0000 7fda08af98c0 1 < canceled operation key=Z2JQBN84QQIB tag=72YQLNL1OOBX type=write size=725647 2024-03-04T17:50:49.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.821+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=TLA7GQBWAO6D type=del size=272750 2024-03-04T17:50:49.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.821+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~IROL7E2364KX.0 size=1048576 2024-03-04T17:50:49.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.825+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~IROL7E2364KX.1 size=853385 2024-03-04T17:50:49.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.829+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=IROL7E2364KX tag=8C9E4KKAPROB type=write size=2950537 2024-03-04T17:50:49.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.833+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=N0OJCLUVVKG4 type=del size=1119836 2024-03-04T17:50:49.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.837+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=E9U2C0S1G23Y type=del 2024-03-04T17:50:49.842 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.845+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~RY7KN1P1O1TK.0 size=1048576 2024-03-04T17:50:49.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.845+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~RY7KN1P1O1TK.1 size=1048576 2024-03-04T17:50:49.848 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.849+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~RY7KN1P1O1TK.2 size=1013585 2024-03-04T17:50:49.848 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.849+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=RY7KN1P1O1TK tag=R4HOP8LMXABF type=write size=4159313 2024-03-04T17:50:49.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.853+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~BD5P87YFUE29.0 size=1048576 2024-03-04T17:50:49.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.857+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~BD5P87YFUE29.1 size=1048576 2024-03-04T17:50:49.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.861+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~BD5P87YFUE29.2 size=811467 2024-03-04T17:50:49.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.865+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=BD5P87YFUE29 tag=XVEK3QDTIE0F type=write size=3957195 2024-03-04T17:50:49.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.873+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=M915DA229Z3V type=del size=1309603 2024-03-04T17:50:49.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.877+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~56KA0VMMD08Q.0 size=620939 2024-03-04T17:50:49.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.881+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=56KA0VMMD08Q tag=UAUTFRLCX27E type=write size=1669515 2024-03-04T17:50:49.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.885+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=80F7LJ3N2ETX type=del size=1799517 2024-03-04T17:50:49.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.889+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=FWZEL4G2QXL8 type=del 2024-03-04T17:50:49.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.897+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~787IZCB05C6G.0 size=1048576 2024-03-04T17:50:49.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.901+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~787IZCB05C6G.1 size=687843 2024-03-04T17:50:49.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.901+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=787IZCB05C6G tag=OTE6MT2W5WCU type=write size=2784995 2024-03-04T17:50:49.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.905+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=FPV3ULIT4FWV type=del 2024-03-04T17:50:49.906 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.909+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=182SJI78Z8IG type=del size=3204989 2024-03-04T17:50:49.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.913+0000 7fda08af98c0 1 < completed write operation key=HCKJZUMAK0Z0 tag=FIR8UMCUOXMI type=write size=427044 2024-03-04T17:50:49.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.917+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~0Q54VZBQCETP.0 size=174569 2024-03-04T17:50:49.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.917+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=0Q54VZBQCETP tag=MUEKN5172ZH8 type=write size=1223145 2024-03-04T17:50:49.920 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.921+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=7AFLKG5TVJ17 type=del 2024-03-04T17:50:49.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.929+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~WBEDQ63WXX3B.0 size=1048576 2024-03-04T17:50:49.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.933+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~WBEDQ63WXX3B.1 size=70167 2024-03-04T17:50:49.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.933+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=WBEDQ63WXX3B tag=LY60BNEQB7G9 type=write size=2167319 2024-03-04T17:50:49.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.937+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~OPSVIRG3097O.0 size=1048576 2024-03-04T17:50:49.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.937+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~OPSVIRG3097O.1 size=1048576 2024-03-04T17:50:49.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.941+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~OPSVIRG3097O.2 size=785929 2024-03-04T17:50:49.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.949+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=OPSVIRG3097O tag=I6WXK08KGZVX type=write size=3931657 2024-03-04T17:50:49.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.953+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~6QMU5RV5MB7X.0 size=538656 2024-03-04T17:50:49.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.953+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=6QMU5RV5MB7X tag=1DT9XKE4YWNL type=write size=1587232 2024-03-04T17:50:49.955 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.957+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=3XOXYIQC5UKX type=del 2024-03-04T17:50:49.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.961+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=ZUGDJCSR5ION type=del size=4105735 2024-03-04T17:50:49.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.965+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=PZKX55R0Q1MF type=del 2024-03-04T17:50:49.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.973+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~K39PTBNFIKA5.0 size=794450 2024-03-04T17:50:49.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.973+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=K39PTBNFIKA5 tag=28BRNRV8CNG2 type=write size=1843026 2024-03-04T17:50:49.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.977+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=233BDK7T771R type=del 2024-03-04T17:50:49.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.981+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=TW0UFFAVJ14R type=del size=3320192 2024-03-04T17:50:49.982 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.985+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~06PQL55QX2JT.0 size=770442 2024-03-04T17:50:49.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.989+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=06PQL55QX2JT tag=T05J5VIKOIB6 type=write size=1819018 2024-03-04T17:50:49.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.993+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=S24HAP7YKI3A type=del size=3333453 2024-03-04T17:50:49.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:49.997+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~EHOLJLBA8C62.0 size=1048576 2024-03-04T17:50:49.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.001+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~EHOLJLBA8C62.1 size=450679 2024-03-04T17:50:50.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.009+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=EHOLJLBA8C62 tag=BTW1P27FCQJL type=write size=2547831 2024-03-04T17:50:50.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.013+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~XUZ60U5ATGYX.0 size=94500 2024-03-04T17:50:50.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.013+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=XUZ60U5ATGYX tag=BW92YRKLQVP3 type=write size=1143076 2024-03-04T17:50:50.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.017+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~JJ1KWZ18BE6L.0 size=1048576 2024-03-04T17:50:50.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.017+0000 7fda08af98c0 1 < canceled part key=_multipart_C4B45GL40IMZ.2~JJ1KWZ18BE6L.1 size=1048576 2024-03-04T17:50:50.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.021+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~JJ1KWZ18BE6L.2 size=653968 2024-03-04T17:50:50.027 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.029+0000 7fda08af98c0 1 < canceled multipart upload key=C4B45GL40IMZ upload=JJ1KWZ18BE6L tag=HG4CLFG14FBI type=write size=3799696 2024-03-04T17:50:50.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.033+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=CW0FW9B1UU27 type=del size=2198956 2024-03-04T17:50:50.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.041+0000 7fda08af98c0 1 < suggested remove operation key=XG2XYBCMUNY5 tag=TLA7GQBWAO6D type=del 2024-03-04T17:50:50.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.045+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=8QZM4DSJKZ7E type=del size=3677436 2024-03-04T17:50:50.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.049+0000 7fda08af98c0 1 < canceled operation key=BYQFZSJ3WH07 tag=N0OJCLUVVKG4 type=del size=1119836 2024-03-04T17:50:50.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.053+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~RXVG0RWD1IY3.0 size=1048576 2024-03-04T17:50:50.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.057+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~RXVG0RWD1IY3.1 size=933892 2024-03-04T17:50:50.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.057+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=RXVG0RWD1IY3 tag=QM0ZCS8I27W7 type=write size=3031044 2024-03-04T17:50:50.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.061+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~RY7KN1P1O1TK.0 size=1048576 2024-03-04T17:50:50.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.065+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~RY7KN1P1O1TK.1 size=1048576 2024-03-04T17:50:50.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.069+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~RY7KN1P1O1TK.2 size=1013585 2024-03-04T17:50:50.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.077+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=RY7KN1P1O1TK tag=R4HOP8LMXABF type=write size=4159313 2024-03-04T17:50:50.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.081+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~T2W01TCRYTOQ.0 size=187016 2024-03-04T17:50:50.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.081+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=T2W01TCRYTOQ tag=QN2YADVBSD54 type=write size=1235592 2024-03-04T17:50:50.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.089+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=M915DA229Z3V type=del 2024-03-04T17:50:50.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.089+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=GA3AMPKAC585 type=del size=3394244 2024-03-04T17:50:50.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.093+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=80F7LJ3N2ETX type=del 2024-03-04T17:50:50.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.097+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=E4EH54RB6BH2 type=del size=435322 2024-03-04T17:50:50.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.101+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~787IZCB05C6G.0 size=1048576 2024-03-04T17:50:50.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.105+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~787IZCB05C6G.1 size=687843 2024-03-04T17:50:50.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.113+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=787IZCB05C6G tag=OTE6MT2W5WCU type=write size=2784995 2024-03-04T17:50:50.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.117+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=QGBHDZHA306H type=del size=1289284 2024-03-04T17:50:50.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.125+0000 7fda08af98c0 1 < suggested update operation key=3UWSL7NJLACY tag=182SJI78Z8IG type=del size=3204989 2024-03-04T17:50:50.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.129+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=CZZZM93VF7SX type=write size=639572 2024-03-04T17:50:50.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.133+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~0Q54VZBQCETP.0 size=174569 2024-03-04T17:50:50.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.137+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=0Q54VZBQCETP tag=MUEKN5172ZH8 type=write size=1223145 2024-03-04T17:50:50.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.141+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=MOYX8CMJJJXE type=del size=3815644 2024-03-04T17:50:50.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.145+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~WBEDQ63WXX3B.0 size=1048576 2024-03-04T17:50:50.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.149+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~WBEDQ63WXX3B.1 size=70167 2024-03-04T17:50:50.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.153+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=WBEDQ63WXX3B tag=LY60BNEQB7G9 type=write size=2167319 2024-03-04T17:50:50.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.157+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=KKC4D1DS7UYH type=write size=106747 2024-03-04T17:50:50.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.161+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~6QMU5RV5MB7X.0 size=538656 2024-03-04T17:50:50.168 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.169+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=6QMU5RV5MB7X tag=1DT9XKE4YWNL type=write size=1587232 2024-03-04T17:50:50.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.169+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=924RMR5DYNDW type=del size=959951 2024-03-04T17:50:50.176 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.177+0000 7fda08af98c0 1 < suggested update operation key=G0BHMSCO0XLF tag=ZUGDJCSR5ION type=del size=4105735 2024-03-04T17:50:50.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.181+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=5XMQU3ONREFC type=del size=3555978 2024-03-04T17:50:50.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.185+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~K39PTBNFIKA5.0 size=794450 2024-03-04T17:50:50.190 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.193+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=K39PTBNFIKA5 tag=28BRNRV8CNG2 type=write size=1843026 2024-03-04T17:50:50.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.197+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=3L03I4VA9KOS type=del size=3905567 2024-03-04T17:50:50.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.197+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=TW0UFFAVJ14R type=del 2024-03-04T17:50:50.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.201+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=L08SBSFW1J4O type=del size=59755 2024-03-04T17:50:50.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.205+0000 7fda08af98c0 -1 < failed to complete operation key=G0BHMSCO0XLF tag=S24HAP7YKI3A type=del size=3333453: (22) Invalid argument 2024-03-04T17:50:50.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.209+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=FA9AEGXJ045V type=write size=796139 2024-03-04T17:50:50.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.213+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~XUZ60U5ATGYX.0 size=94500 2024-03-04T17:50:50.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.221+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=XUZ60U5ATGYX tag=BW92YRKLQVP3 type=write size=1143076 2024-03-04T17:50:50.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.225+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=PJBKX5XRC9MN type=del size=3764990 2024-03-04T17:50:50.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.233+0000 7fda08af98c0 1 < suggested remove operation key=2PA8JDZUOLO5 tag=CW0FW9B1UU27 type=del 2024-03-04T17:50:50.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.237+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~TITRJZ4PP2T6.0 size=275400 2024-03-04T17:50:50.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.237+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=TITRJZ4PP2T6 tag=UZ9OPY3J2L3K type=write size=1323976 2024-03-04T17:50:50.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.241+0000 7fda08af98c0 -1 < failed to complete operation key=G0BHMSCO0XLF tag=8QZM4DSJKZ7E type=del size=3677436: (22) Invalid argument 2024-03-04T17:50:50.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.245+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=5H35JUXCXGLO type=del size=3596757 2024-03-04T17:50:50.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.249+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~RXVG0RWD1IY3.0 size=1048576 2024-03-04T17:50:50.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.253+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~RXVG0RWD1IY3.1 size=933892 2024-03-04T17:50:50.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.261+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=RXVG0RWD1IY3 tag=QM0ZCS8I27W7 type=write size=3031044 2024-03-04T17:50:50.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.261+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=87VIH5YC9IMT type=del size=1794626 2024-03-04T17:50:50.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.265+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~T2W01TCRYTOQ.0 size=187016 2024-03-04T17:50:50.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.273+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=T2W01TCRYTOQ tag=QN2YADVBSD54 type=write size=1235592 2024-03-04T17:50:50.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.281+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~YIA2WHOHPKPQ.0 size=1048576 2024-03-04T17:50:50.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.281+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~YIA2WHOHPKPQ.1 size=1048576 2024-03-04T17:50:50.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.285+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~YIA2WHOHPKPQ.2 size=380299 2024-03-04T17:50:50.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.285+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=YIA2WHOHPKPQ tag=YR5I7GA8AYAZ type=write size=3526027 2024-03-04T17:50:50.288 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.289+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=GA3AMPKAC585 type=del 2024-03-04T17:50:50.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.297+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~6LYRJV0HDVFT.0 size=1048576 2024-03-04T17:50:50.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.297+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~6LYRJV0HDVFT.1 size=264151 2024-03-04T17:50:50.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.297+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=6LYRJV0HDVFT tag=FHHIBX6U50UY type=write size=2361303 2024-03-04T17:50:50.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.301+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=E4EH54RB6BH2 type=del 2024-03-04T17:50:50.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.305+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=B205ZH1GOUYM type=del size=24227 2024-03-04T17:50:50.308 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.309+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=QGBHDZHA306H type=del 2024-03-04T17:50:50.312 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.313+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=6W0M4FFC9AX5 type=del size=2687141 2024-03-04T17:50:50.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.317+0000 7fda08af98c0 1 < completed write operation key=IJSYAVR5SWPS tag=CZZZM93VF7SX type=write size=639572 2024-03-04T17:50:50.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.321+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=H6OCPTAOX2K3 type=del size=2607915 2024-03-04T17:50:50.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.325+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=MOYX8CMJJJXE type=del 2024-03-04T17:50:50.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.329+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=GKHM4VCF18YD type=del size=3803533 2024-03-04T17:50:50.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.333+0000 7fda08af98c0 1 < completed write operation key=FNWA2N9U14HO tag=KKC4D1DS7UYH type=write size=106747 2024-03-04T17:50:50.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.337+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=CH3B4ZL8S949 type=del size=3176750 2024-03-04T17:50:50.340 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.341+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=924RMR5DYNDW type=del 2024-03-04T17:50:50.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.345+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=D4TH3WNBMYVM type=del size=3507588 2024-03-04T17:50:50.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.349+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=5XMQU3ONREFC type=del 2024-03-04T17:50:50.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.353+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=WEP1FQL4LAS3 type=del size=3744627 2024-03-04T17:50:50.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.357+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=3L03I4VA9KOS type=del 2024-03-04T17:50:50.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.361+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=FHPL8KJBLRE0 type=del size=2242667 2024-03-04T17:50:50.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.365+0000 7fda08af98c0 1 < canceled operation key=ITIVF3UAT9NK tag=L08SBSFW1J4O type=del size=59755 2024-03-04T17:50:50.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.369+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=KOKW84G6US7B type=del size=3811852 2024-03-04T17:50:50.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.373+0000 7fda08af98c0 1 < completed write operation key=3UWSL7NJLACY tag=FA9AEGXJ045V type=write size=796139 2024-03-04T17:50:50.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.377+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=HSLDS4X1VJCW type=del size=2654540 2024-03-04T17:50:50.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.389+0000 7fda08af98c0 1 < suggested remove operation key=FNWA2N9U14HO tag=PJBKX5XRC9MN type=del 2024-03-04T17:50:50.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.393+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~ANO6Y98G4VEZ.0 size=1048576 2024-03-04T17:50:50.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.397+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~ANO6Y98G4VEZ.1 size=1019355 2024-03-04T17:50:50.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.397+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=ANO6Y98G4VEZ tag=G0SS5021G53E type=write size=3116507 2024-03-04T17:50:50.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.401+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~TITRJZ4PP2T6.0 size=275400 2024-03-04T17:50:50.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.409+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=TITRJZ4PP2T6 tag=UZ9OPY3J2L3K type=write size=1323976 2024-03-04T17:50:50.410 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.413+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=K756IPD1DKIJ type=del size=3062215 2024-03-04T17:50:50.414 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.417+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=5H35JUXCXGLO type=del 2024-03-04T17:50:50.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.421+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=2JBJ892DWENO type=del size=2044902 2024-03-04T17:50:50.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.429+0000 7fda08af98c0 1 < suggested update operation key=VJVORUG95FLC tag=87VIH5YC9IMT type=del size=1794626 2024-03-04T17:50:50.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.433+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=KYU4Q5H7E30B type=del size=425909 2024-03-04T17:50:50.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.437+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~YIA2WHOHPKPQ.0 size=1048576 2024-03-04T17:50:50.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.441+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~YIA2WHOHPKPQ.1 size=1048576 2024-03-04T17:50:50.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.445+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~YIA2WHOHPKPQ.2 size=380299 2024-03-04T17:50:50.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.453+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=YIA2WHOHPKPQ tag=YR5I7GA8AYAZ type=write size=3526027 2024-03-04T17:50:50.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.457+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=LU6ZX17GCIJJ type=del size=867494 2024-03-04T17:50:50.459 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.461+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~6LYRJV0HDVFT.0 size=1048576 2024-03-04T17:50:50.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.465+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~6LYRJV0HDVFT.1 size=264151 2024-03-04T17:50:50.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.469+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=6LYRJV0HDVFT tag=FHHIBX6U50UY type=write size=2361303 2024-03-04T17:50:50.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.477+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~PRJUL2V7VLVM.0 size=1048576 2024-03-04T17:50:50.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.481+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~PRJUL2V7VLVM.1 size=1048576 2024-03-04T17:50:50.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.481+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~PRJUL2V7VLVM.2 size=638252 2024-03-04T17:50:50.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.481+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=PRJUL2V7VLVM tag=VPFS9M3GXTX5 type=write size=3783980 2024-03-04T17:50:50.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.489+0000 7fda08af98c0 1 < suggested update operation key=VJVORUG95FLC tag=B205ZH1GOUYM type=del size=24227 2024-03-04T17:50:50.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.493+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=UAU0AOJQV2E5 type=write size=820453 2024-03-04T17:50:50.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.501+0000 7fda08af98c0 1 < suggested remove operation key=8YL7C9PI58TK tag=6W0M4FFC9AX5 type=del 2024-03-04T17:50:50.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.505+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=QH1GGOKWOYMQ type=del size=287237 2024-03-04T17:50:50.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.509+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=H6OCPTAOX2K3 type=del 2024-03-04T17:50:50.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.513+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=CZF03MT3THKH type=write size=330597 2024-03-04T17:50:50.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.521+0000 7fda08af98c0 1 < suggested update operation key=XG5I3SDRE5JM tag=GKHM4VCF18YD type=del size=3803533 2024-03-04T17:50:50.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.521+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=6MR1KN2BBXYQ type=del size=4194127 2024-03-04T17:50:50.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.525+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=CH3B4ZL8S949 type=del 2024-03-04T17:50:50.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.529+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=KSTJRTD0VAOG type=write size=804261 2024-03-04T17:50:50.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.533+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=D4TH3WNBMYVM type=del 2024-03-04T17:50:50.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.541+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~3GOL8RDNVW4E.0 size=644859 2024-03-04T17:50:50.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.541+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=3GOL8RDNVW4E tag=DA83J7BYYO2O type=write size=1693435 2024-03-04T17:50:50.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.545+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=WEP1FQL4LAS3 type=del 2024-03-04T17:50:50.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.549+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=TB455BSD0FSZ type=del size=617462 2024-03-04T17:50:50.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.553+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=FHPL8KJBLRE0 type=del 2024-03-04T17:50:50.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.557+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=NUAXYRYWHQNH type=del size=2034487 2024-03-04T17:50:50.559 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.561+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=KOKW84G6US7B type=del 2024-03-04T17:50:50.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.569+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~RY6E5983P05F.0 size=1048576 2024-03-04T17:50:50.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.569+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~RY6E5983P05F.1 size=1048576 2024-03-04T17:50:50.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.573+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~RY6E5983P05F.2 size=369302 2024-03-04T17:50:50.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.573+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=RY6E5983P05F tag=ODVZNYISD7UW type=write size=3515030 2024-03-04T17:50:50.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.577+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=HSLDS4X1VJCW type=del 2024-03-04T17:50:50.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.581+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=32D0GCHH1T21 type=del size=1909872 2024-03-04T17:50:50.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.589+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~ANO6Y98G4VEZ.0 size=1048576 2024-03-04T17:50:50.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.593+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~ANO6Y98G4VEZ.1 size=1019355 2024-03-04T17:50:50.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.597+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=ANO6Y98G4VEZ tag=G0SS5021G53E type=write size=3116507 2024-03-04T17:50:50.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.605+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=1JVGIRR2RWT8 type=del size=353218 2024-03-04T17:50:50.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.609+0000 7fda08af98c0 -1 < failed to complete operation key=VJVORUG95FLC tag=K756IPD1DKIJ type=del size=3062215: (22) Invalid argument 2024-03-04T17:50:50.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.613+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=XA2KQ58JMLKF type=del size=1716830 2024-03-04T17:50:50.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.617+0000 7fda08af98c0 1 < canceled operation key=L0FVKWD7EQRW tag=2JBJ892DWENO type=del size=2044902 2024-03-04T17:50:50.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.621+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=6SRW8O18HD41 type=del size=2681491 2024-03-04T17:50:50.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.625+0000 7fda08af98c0 -1 < failed to complete operation key=XG5I3SDRE5JM tag=KYU4Q5H7E30B type=del size=425909: (22) Invalid argument 2024-03-04T17:50:50.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.633+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~QYRGV1APB2Q9.0 size=1048576 2024-03-04T17:50:50.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.633+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~QYRGV1APB2Q9.1 size=389861 2024-03-04T17:50:50.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.633+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=QYRGV1APB2Q9 tag=X11P2X5RWJ5K type=write size=2487013 2024-03-04T17:50:50.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.641+0000 7fda08af98c0 1 < suggested remove operation key=Z2JQBN84QQIB tag=LU6ZX17GCIJJ type=del 2024-03-04T17:50:50.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.645+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=TXKYW4DC9MLG type=del size=1007284 2024-03-04T17:50:50.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.649+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~PRJUL2V7VLVM.0 size=1048576 2024-03-04T17:50:50.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.653+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~PRJUL2V7VLVM.1 size=1048576 2024-03-04T17:50:50.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.653+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~PRJUL2V7VLVM.2 size=638252 2024-03-04T17:50:50.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.661+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=PRJUL2V7VLVM tag=VPFS9M3GXTX5 type=write size=3783980 2024-03-04T17:50:50.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.665+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=VIS4OIO72YE9 type=del size=1618238 2024-03-04T17:50:50.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.669+0000 7fda08af98c0 1 < completed write operation key=RXVSQVVNXVF0 tag=UAU0AOJQV2E5 type=write size=820453 2024-03-04T17:50:50.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.673+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=DAKWNDKC2CHS type=del size=2612062 2024-03-04T17:50:50.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.677+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=QH1GGOKWOYMQ type=del 2024-03-04T17:50:50.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.681+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=R6VY5R16726B type=del size=1076246 2024-03-04T17:50:50.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.689+0000 7fda08af98c0 1 < suggested update operation key=M5BMX7B7RZYL tag=CZF03MT3THKH type=write size=330597 2024-03-04T17:50:50.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.693+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~HI2TARD3PMX8.0 size=1048576 2024-03-04T17:50:50.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.697+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~HI2TARD3PMX8.1 size=1048576 2024-03-04T17:50:50.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.701+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~HI2TARD3PMX8.2 size=1027940 2024-03-04T17:50:50.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.701+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=HI2TARD3PMX8 tag=BSQ3PEVS38I2 type=write size=4173668 2024-03-04T17:50:50.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.705+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=6MR1KN2BBXYQ type=del 2024-03-04T17:50:50.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.709+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~XD97ADMYTD54.0 size=546611 2024-03-04T17:50:50.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.709+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=XD97ADMYTD54 tag=UNM9L01EZU6V type=write size=1595187 2024-03-04T17:50:50.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.713+0000 7fda08af98c0 1 < completed write operation key=XG2XYBCMUNY5 tag=KSTJRTD0VAOG type=write size=804261 2024-03-04T17:50:50.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.721+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~T6DIWJXG40TH.0 size=1048576 2024-03-04T17:50:50.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.725+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~T6DIWJXG40TH.1 size=1048576 2024-03-04T17:50:50.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.729+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~T6DIWJXG40TH.2 size=752963 2024-03-04T17:50:50.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.729+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=T6DIWJXG40TH tag=3SAQLY6MBZMY type=write size=3898691 2024-03-04T17:50:50.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.733+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~3GOL8RDNVW4E.0 size=644859 2024-03-04T17:50:50.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.737+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=3GOL8RDNVW4E tag=DA83J7BYYO2O type=write size=1693435 2024-03-04T17:50:50.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.741+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=HVA0HCEA2X21 type=write size=920979 2024-03-04T17:50:50.747 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.749+0000 7fda08af98c0 1 < suggested remove operation key=XG2XYBCMUNY5 tag=TB455BSD0FSZ type=del 2024-03-04T17:50:50.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.753+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=TTORGJRZDSA3 type=write size=420609 2024-03-04T17:50:50.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.761+0000 7fda08af98c0 1 < suggested update operation key=FMPZ2548LBBF tag=NUAXYRYWHQNH type=del size=2034487 2024-03-04T17:50:50.762 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.765+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=GG2JK4KGZNT1 type=write size=252403 2024-03-04T17:50:50.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.769+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~RY6E5983P05F.0 size=1048576 2024-03-04T17:50:50.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.773+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~RY6E5983P05F.1 size=1048576 2024-03-04T17:50:50.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.773+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~RY6E5983P05F.2 size=369302 2024-03-04T17:50:50.780 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.781+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=RY6E5983P05F tag=ODVZNYISD7UW type=write size=3515030 2024-03-04T17:50:50.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.785+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=SKSXFPVY88TM type=write size=503099 2024-03-04T17:50:50.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.789+0000 7fda08af98c0 -1 < failed to complete operation key=XG2XYBCMUNY5 tag=32D0GCHH1T21 type=del size=1909872: (22) Invalid argument 2024-03-04T17:50:50.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.797+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=SVV8DGA838YP type=del size=3011040 2024-03-04T17:50:50.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.801+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=1JVGIRR2RWT8 type=del 2024-03-04T17:50:50.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.805+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=1Q8T4G9NMWXB type=write size=733950 2024-03-04T17:50:50.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.817+0000 7fda08af98c0 1 < suggested remove operation key=FNWA2N9U14HO tag=XA2KQ58JMLKF type=del 2024-03-04T17:50:50.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.817+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=M1UOL0O8MXXU type=del size=1837044 2024-03-04T17:50:50.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.821+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=6SRW8O18HD41 type=del 2024-03-04T17:50:50.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.825+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=NHRBET84ACSK type=del size=2933231 2024-03-04T17:50:50.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.829+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~QYRGV1APB2Q9.0 size=1048576 2024-03-04T17:50:50.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.833+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~QYRGV1APB2Q9.1 size=389861 2024-03-04T17:50:50.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.841+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=QYRGV1APB2Q9 tag=X11P2X5RWJ5K type=write size=2487013 2024-03-04T17:50:50.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.845+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=T6E2OL5AWQXY type=write size=242531 2024-03-04T17:50:50.848 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.849+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=TXKYW4DC9MLG type=del 2024-03-04T17:50:50.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.857+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~702V01S4CM7Z.0 size=354572 2024-03-04T17:50:50.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.857+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=702V01S4CM7Z tag=MVCLP80MQIGY type=write size=1403148 2024-03-04T17:50:50.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.861+0000 7fda08af98c0 -1 < failed to complete operation key=M5BMX7B7RZYL tag=VIS4OIO72YE9 type=del size=1618238: (22) Invalid argument 2024-03-04T17:50:50.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.865+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=PLJO5ND6ZODS type=del size=1147416 2024-03-04T17:50:50.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.869+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=DAKWNDKC2CHS type=del 2024-03-04T17:50:50.875 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.877+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~DVUO0UMKLS6J.0 size=365427 2024-03-04T17:50:50.875 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.877+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=DVUO0UMKLS6J tag=CL2NLRN2KBZJ type=write size=1414003 2024-03-04T17:50:50.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.881+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=R6VY5R16726B type=del 2024-03-04T17:50:50.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.885+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=A3NRKQRS2827 type=del size=1127230 2024-03-04T17:50:50.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.889+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~HI2TARD3PMX8.0 size=1048576 2024-03-04T17:50:50.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.893+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~HI2TARD3PMX8.1 size=1048576 2024-03-04T17:50:50.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.897+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~HI2TARD3PMX8.2 size=1027940 2024-03-04T17:50:50.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.905+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=HI2TARD3PMX8 tag=BSQ3PEVS38I2 type=write size=4173668 2024-03-04T17:50:50.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.909+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~R893ZY4YDQUL.0 size=1048576 2024-03-04T17:50:50.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.913+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~R893ZY4YDQUL.1 size=980879 2024-03-04T17:50:50.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.913+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=R893ZY4YDQUL tag=S5GNKD0TN11Y type=write size=3078031 2024-03-04T17:50:50.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.917+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~XD97ADMYTD54.0 size=546611 2024-03-04T17:50:50.922 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.925+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=XD97ADMYTD54 tag=UNM9L01EZU6V type=write size=1595187 2024-03-04T17:50:50.925 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.929+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=ZXE9JJ101X01 type=del size=424012 2024-03-04T17:50:50.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.933+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~T6DIWJXG40TH.0 size=1048576 2024-03-04T17:50:50.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.933+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~T6DIWJXG40TH.1 size=1048576 2024-03-04T17:50:50.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.937+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~T6DIWJXG40TH.2 size=752963 2024-03-04T17:50:50.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.945+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=T6DIWJXG40TH tag=3SAQLY6MBZMY type=write size=3898691 2024-03-04T17:50:50.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.949+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=UZ8SLYQUNHWV type=del size=4075292 2024-03-04T17:50:50.953 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.953+0000 7fda08af98c0 1 < suggested remove operation key=GDGTJZ21TG32 tag=HVA0HCEA2X21 type=write 2024-03-04T17:50:50.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.961+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=D6RJRSDKAR9W type=del size=3346347 2024-03-04T17:50:50.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.965+0000 7fda08af98c0 1 < completed write operation key=IJSYAVR5SWPS tag=TTORGJRZDSA3 type=write size=420609 2024-03-04T17:50:50.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.969+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~K5OE0YDSXY7W.0 size=1048576 2024-03-04T17:50:50.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.973+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~K5OE0YDSXY7W.1 size=1048576 2024-03-04T17:50:50.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.977+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~K5OE0YDSXY7W.2 size=786208 2024-03-04T17:50:50.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.977+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=K5OE0YDSXY7W tag=BJ91W1T5HJBG type=write size=3931936 2024-03-04T17:50:50.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.981+0000 7fda08af98c0 1 < completed write operation key=XG5I3SDRE5JM tag=GG2JK4KGZNT1 type=write size=252403 2024-03-04T17:50:50.982 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.985+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=8UG2VYB327NY type=write size=250018 2024-03-04T17:50:50.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.989+0000 7fda08af98c0 1 < completed write operation key=M5BMX7B7RZYL tag=SKSXFPVY88TM type=write size=503099 2024-03-04T17:50:50.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.993+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=X9SUR1P4XOES type=del size=479528 2024-03-04T17:50:50.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:50.997+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=SVV8DGA838YP type=del 2024-03-04T17:50:51.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.001+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~RLDXY8420KFC.0 size=747392 2024-03-04T17:50:51.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.001+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=RLDXY8420KFC tag=AQ02VYUBYO2O type=write size=1795968 2024-03-04T17:50:51.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.005+0000 7fda08af98c0 1 < completed write operation key=G0BHMSCO0XLF tag=1Q8T4G9NMWXB type=write size=733950 2024-03-04T17:50:51.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.013+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~Y1QS7AQOSUIH.0 size=1048576 2024-03-04T17:50:51.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.013+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~Y1QS7AQOSUIH.1 size=1048576 2024-03-04T17:50:51.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.017+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~Y1QS7AQOSUIH.2 size=434265 2024-03-04T17:50:51.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.017+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=Y1QS7AQOSUIH tag=JAIS1W9PSSKC type=write size=3579993 2024-03-04T17:50:51.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.021+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=M1UOL0O8MXXU type=del 2024-03-04T17:50:51.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.025+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=M35GYYE5OTQR type=del size=1457864 2024-03-04T17:50:51.027 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.029+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=NHRBET84ACSK type=del 2024-03-04T17:50:51.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.033+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=6HAITL9T7KEV type=del size=4003333 2024-03-04T17:50:51.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.037+0000 7fda08af98c0 1 < completed write operation key=BYQFZSJ3WH07 tag=T6E2OL5AWQXY type=write size=242531 2024-03-04T17:50:51.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.041+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=4VII16EWYVUI type=del size=2775041 2024-03-04T17:50:51.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.045+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~702V01S4CM7Z.0 size=354572 2024-03-04T17:50:51.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.049+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=702V01S4CM7Z tag=MVCLP80MQIGY type=write size=1403148 2024-03-04T17:50:51.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.057+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~ESHT70KIWFYT.0 size=288337 2024-03-04T17:50:51.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.057+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=ESHT70KIWFYT tag=1TF8IAQUAJ9Y type=write size=1336913 2024-03-04T17:50:51.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.061+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=PLJO5ND6ZODS type=del 2024-03-04T17:50:51.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.065+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=QPLCWVXMRURI type=del size=2453506 2024-03-04T17:50:51.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.069+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~DVUO0UMKLS6J.0 size=365427 2024-03-04T17:50:51.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.077+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=DVUO0UMKLS6J tag=CL2NLRN2KBZJ type=write size=1414003 2024-03-04T17:50:51.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.081+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~77CLOK455M9V.0 size=107103 2024-03-04T17:50:51.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.081+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=77CLOK455M9V tag=BBGQHK2GY2UK type=write size=1155679 2024-03-04T17:50:51.084 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.085+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=A3NRKQRS2827 type=del 2024-03-04T17:50:51.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.089+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=68CCFX9F0L3R type=del size=822787 2024-03-04T17:50:51.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.093+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~R893ZY4YDQUL.0 size=1048576 2024-03-04T17:50:51.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.097+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~R893ZY4YDQUL.1 size=980879 2024-03-04T17:50:51.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.105+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=R893ZY4YDQUL tag=S5GNKD0TN11Y type=write size=3078031 2024-03-04T17:50:51.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.109+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=NZ2HNA56IYFB type=del size=1086478 2024-03-04T17:50:51.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.113+0000 7fda08af98c0 1 < suggested update operation key=3UWSL7NJLACY tag=ZXE9JJ101X01 type=del size=424012 2024-03-04T17:50:51.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.117+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=2A8SA54AUAHD type=del size=2707285 2024-03-04T17:50:51.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.125+0000 7fda08af98c0 1 < suggested update operation key=BYQFZSJ3WH07 tag=UZ8SLYQUNHWV type=del size=4075292 2024-03-04T17:50:51.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.133+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~EXULSBXDUFGT.0 size=1048576 2024-03-04T17:50:51.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.137+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~EXULSBXDUFGT.1 size=598078 2024-03-04T17:50:51.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.137+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=EXULSBXDUFGT tag=OAJXSK1EX0CO type=write size=2695230 2024-03-04T17:50:51.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.141+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=D6RJRSDKAR9W type=del 2024-03-04T17:50:51.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.145+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=X972IYAW0HFQ type=del size=2659674 2024-03-04T17:50:51.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.149+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~K5OE0YDSXY7W.0 size=1048576 2024-03-04T17:50:51.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.153+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~K5OE0YDSXY7W.1 size=1048576 2024-03-04T17:50:51.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.157+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~K5OE0YDSXY7W.2 size=786208 2024-03-04T17:50:51.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.165+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=K5OE0YDSXY7W tag=BJ91W1T5HJBG type=write size=3931936 2024-03-04T17:50:51.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.173+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~ESLN9LUX4ICC.0 size=1048576 2024-03-04T17:50:51.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.177+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~ESLN9LUX4ICC.1 size=527702 2024-03-04T17:50:51.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.177+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=ESLN9LUX4ICC tag=Q6KN51NIOGV5 type=write size=2624854 2024-03-04T17:50:51.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.181+0000 7fda08af98c0 1 < completed write operation key=XG2XYBCMUNY5 tag=8UG2VYB327NY type=write size=250018 2024-03-04T17:50:51.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.185+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=PA934FF9G3TK type=write size=1021666 2024-03-04T17:50:51.187 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.189+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=X9SUR1P4XOES type=del 2024-03-04T17:50:51.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.197+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~8EGRJLIG6DD3.0 size=1048576 2024-03-04T17:50:51.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.197+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~8EGRJLIG6DD3.1 size=877447 2024-03-04T17:50:51.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.197+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=8EGRJLIG6DD3 tag=E2ALS76YDN1B type=write size=2974599 2024-03-04T17:50:51.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.201+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~RLDXY8420KFC.0 size=747392 2024-03-04T17:50:51.206 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.209+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=RLDXY8420KFC tag=AQ02VYUBYO2O type=write size=1795968 2024-03-04T17:50:51.214 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.217+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~MUFJ1VVP2K5X.0 size=1048576 2024-03-04T17:50:51.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.217+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~MUFJ1VVP2K5X.1 size=494051 2024-03-04T17:50:51.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.217+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=MUFJ1VVP2K5X tag=C52JAFF8AIKR type=write size=2591203 2024-03-04T17:50:51.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.221+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~Y1QS7AQOSUIH.0 size=1048576 2024-03-04T17:50:51.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.225+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~Y1QS7AQOSUIH.1 size=1048576 2024-03-04T17:50:51.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.229+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~Y1QS7AQOSUIH.2 size=434265 2024-03-04T17:50:51.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.233+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=Y1QS7AQOSUIH tag=JAIS1W9PSSKC type=write size=3579993 2024-03-04T17:50:51.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.237+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=Y6I9ACD1NYE4 type=del size=1055758 2024-03-04T17:50:51.245 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.245+0000 7fda08af98c0 1 < suggested remove operation key=Z3J7W9V8HQKH tag=M35GYYE5OTQR type=del 2024-03-04T17:50:51.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.249+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=L5V7MZSBP5U1 type=del size=859575 2024-03-04T17:50:51.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.253+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=6HAITL9T7KEV type=del 2024-03-04T17:50:51.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.257+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=S7BI1BCXDRGE type=del size=731357 2024-03-04T17:50:51.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.261+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=4VII16EWYVUI type=del 2024-03-04T17:50:51.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.269+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~LJA3ZPAEOSAG.0 size=427350 2024-03-04T17:50:51.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.269+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=LJA3ZPAEOSAG tag=YORZX549TRUR type=write size=1475926 2024-03-04T17:50:51.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.273+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~ESHT70KIWFYT.0 size=288337 2024-03-04T17:50:51.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.277+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=ESHT70KIWFYT tag=1TF8IAQUAJ9Y type=write size=1336913 2024-03-04T17:50:51.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.281+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=2DHC8YGHVV30 type=del size=415894 2024-03-04T17:50:51.288 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.289+0000 7fda08af98c0 1 < suggested remove operation key=XG2XYBCMUNY5 tag=QPLCWVXMRURI type=del 2024-03-04T17:50:51.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.293+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=IIK1N7L2ZOX1 type=del size=3336302 2024-03-04T17:50:51.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.297+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~77CLOK455M9V.0 size=107103 2024-03-04T17:50:51.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.305+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=77CLOK455M9V tag=BBGQHK2GY2UK type=write size=1155679 2024-03-04T17:50:51.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.309+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=DQFRTO447NUC type=write size=714980 2024-03-04T17:50:51.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.313+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=68CCFX9F0L3R type=del 2024-03-04T17:50:51.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.317+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=FXUTXQX6VRJJ type=del size=2088270 2024-03-04T17:50:51.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.321+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=NZ2HNA56IYFB type=del 2024-03-04T17:50:51.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.329+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~KES396HUR7DM.0 size=1048576 2024-03-04T17:50:51.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.333+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~KES396HUR7DM.1 size=707178 2024-03-04T17:50:51.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.333+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=KES396HUR7DM tag=L4N9J5DXO5BL type=write size=2804330 2024-03-04T17:50:51.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.341+0000 7fda08af98c0 1 < suggested remove operation key=1OZKPHMPV3OQ tag=2A8SA54AUAHD type=del 2024-03-04T17:50:51.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.345+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~9Q4OCQGVLJUW.0 size=627004 2024-03-04T17:50:51.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.345+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=9Q4OCQGVLJUW tag=T2UD8SRWP3FX type=write size=1675580 2024-03-04T17:50:51.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.349+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~EXULSBXDUFGT.0 size=1048576 2024-03-04T17:50:51.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.349+0000 7fda08af98c0 1 < canceled part key=_multipart_HCKJZUMAK0Z0.2~EXULSBXDUFGT.1 size=598078 2024-03-04T17:50:51.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.357+0000 7fda08af98c0 1 < canceled multipart upload key=HCKJZUMAK0Z0 upload=EXULSBXDUFGT tag=OAJXSK1EX0CO type=write size=2695230 2024-03-04T17:50:51.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.361+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=1ALY6YU86RXQ type=write size=547338 2024-03-04T17:50:51.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.365+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=X972IYAW0HFQ type=del 2024-03-04T17:50:51.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.373+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~UYVPMO6FYK4R.0 size=1048576 2024-03-04T17:50:51.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.373+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~UYVPMO6FYK4R.1 size=1048576 2024-03-04T17:50:51.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.377+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~UYVPMO6FYK4R.2 size=1703 2024-03-04T17:50:51.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.377+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=UYVPMO6FYK4R tag=S2U5GQ5562EF type=write size=3147431 2024-03-04T17:50:51.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.381+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~ESLN9LUX4ICC.0 size=1048576 2024-03-04T17:50:51.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.385+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~ESLN9LUX4ICC.1 size=527702 2024-03-04T17:50:51.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.393+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=ESLN9LUX4ICC tag=Q6KN51NIOGV5 type=write size=2624854 2024-03-04T17:50:51.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.397+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=DPG9O5Q96Y52 type=del size=2064704 2024-03-04T17:50:51.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.401+0000 7fda08af98c0 1 < completed write operation key=2PA8JDZUOLO5 tag=PA934FF9G3TK type=write size=1021666 2024-03-04T17:50:51.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.405+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=RS3CXOFEPE2M type=del size=1025390 2024-03-04T17:50:51.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.409+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~8EGRJLIG6DD3.0 size=1048576 2024-03-04T17:50:51.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.413+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~8EGRJLIG6DD3.1 size=877447 2024-03-04T17:50:51.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.417+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=8EGRJLIG6DD3 tag=E2ALS76YDN1B type=write size=2974599 2024-03-04T17:50:51.420 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.421+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=44T022921JWT type=del size=314289 2024-03-04T17:50:51.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.425+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~MUFJ1VVP2K5X.0 size=1048576 2024-03-04T17:50:51.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.429+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~MUFJ1VVP2K5X.1 size=494051 2024-03-04T17:50:51.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.437+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=MUFJ1VVP2K5X tag=C52JAFF8AIKR type=write size=2591203 2024-03-04T17:50:51.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.441+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~43WHX51OX4P9.0 size=67881 2024-03-04T17:50:51.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.441+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=43WHX51OX4P9 tag=787IG6V4GZ9L type=write size=1116457 2024-03-04T17:50:51.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.445+0000 7fda08af98c0 1 < canceled operation key=09KBCA2XURTF tag=Y6I9ACD1NYE4 type=del size=1055758 2024-03-04T17:50:51.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.449+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=I84PW6I7I11K type=write size=105410 2024-03-04T17:50:51.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.453+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=L5V7MZSBP5U1 type=del 2024-03-04T17:50:51.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.457+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=88VT5T5HOFMM type=del size=4105155 2024-03-04T17:50:51.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.461+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=S7BI1BCXDRGE type=del 2024-03-04T17:50:51.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.469+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~FQ0FAXDTPP2S.0 size=1048576 2024-03-04T17:50:51.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.473+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~FQ0FAXDTPP2S.1 size=1048576 2024-03-04T17:50:51.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.473+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~FQ0FAXDTPP2S.2 size=697329 2024-03-04T17:50:51.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.473+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=FQ0FAXDTPP2S tag=DJE8B9KWPKST type=write size=3843057 2024-03-04T17:50:51.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.477+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~LJA3ZPAEOSAG.0 size=427350 2024-03-04T17:50:51.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.485+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=LJA3ZPAEOSAG tag=YORZX549TRUR type=write size=1475926 2024-03-04T17:50:51.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.489+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~IH8G200I8SMO.0 size=1048576 2024-03-04T17:50:51.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.493+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~IH8G200I8SMO.1 size=1040229 2024-03-04T17:50:51.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.493+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=IH8G200I8SMO tag=IMEFGPWC5RYS type=write size=3137381 2024-03-04T17:50:51.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.501+0000 7fda08af98c0 1 < suggested remove operation key=IJSYAVR5SWPS tag=2DHC8YGHVV30 type=del 2024-03-04T17:50:51.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.505+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=9YBPSNUK3RD4 type=write size=731313 2024-03-04T17:50:51.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.509+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=IIK1N7L2ZOX1 type=del 2024-03-04T17:50:51.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.513+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=NNFDTNPM2XFO type=del size=2913549 2024-03-04T17:50:51.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.517+0000 7fda08af98c0 -1 < failed to complete operation key=1OZKPHMPV3OQ tag=DQFRTO447NUC type=write size=714980: (22) Invalid argument 2024-03-04T17:50:51.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.521+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=Z3TX0MVKCKN3 type=del size=3197390 2024-03-04T17:50:51.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.533+0000 7fda08af98c0 1 < suggested remove operation key=X5MEJ7G8X9HE tag=FXUTXQX6VRJJ type=del 2024-03-04T17:50:51.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.537+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=VCXA2F8Q35KS type=del size=604814 2024-03-04T17:50:51.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.541+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~KES396HUR7DM.0 size=1048576 2024-03-04T17:50:51.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.545+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~KES396HUR7DM.1 size=707178 2024-03-04T17:50:51.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.549+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=KES396HUR7DM tag=L4N9J5DXO5BL type=write size=2804330 2024-03-04T17:50:51.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.553+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=RT07BSQIIS6D type=del size=1565959 2024-03-04T17:50:51.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.557+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~9Q4OCQGVLJUW.0 size=627004 2024-03-04T17:50:51.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.565+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=9Q4OCQGVLJUW tag=T2UD8SRWP3FX type=write size=1675580 2024-03-04T17:50:51.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.569+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~5NZZNK5XCF1O.0 size=1048576 2024-03-04T17:50:51.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.573+0000 7fda08af98c0 1 > prepared part key=_multipart_0IGSMFORCSL9.2~5NZZNK5XCF1O.1 size=318621 2024-03-04T17:50:51.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.573+0000 7fda08af98c0 1 > prepared multipart upload key=0IGSMFORCSL9 upload=5NZZNK5XCF1O tag=0M33XEQHWAL5 type=write size=2415773 2024-03-04T17:50:51.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.577+0000 7fda08af98c0 1 < completed write operation key=BYQFZSJ3WH07 tag=1ALY6YU86RXQ type=write size=547338 2024-03-04T17:50:51.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.585+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~0I1M7JU3ZNKN.0 size=1048576 2024-03-04T17:50:51.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.585+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~0I1M7JU3ZNKN.1 size=1048576 2024-03-04T17:50:51.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.589+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~0I1M7JU3ZNKN.2 size=378278 2024-03-04T17:50:51.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.589+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=0I1M7JU3ZNKN tag=TV3L7IORQAPM type=write size=3524006 2024-03-04T17:50:51.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.593+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~UYVPMO6FYK4R.0 size=1048576 2024-03-04T17:50:51.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.597+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~UYVPMO6FYK4R.1 size=1048576 2024-03-04T17:50:51.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.601+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~UYVPMO6FYK4R.2 size=1703 2024-03-04T17:50:51.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.605+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=UYVPMO6FYK4R tag=S2U5GQ5562EF type=write size=3147431 2024-03-04T17:50:51.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.609+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=4KIUMZ9I0NK0 type=write size=603901 2024-03-04T17:50:51.615 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.617+0000 7fda08af98c0 1 < suggested update operation key=GDGTJZ21TG32 tag=DPG9O5Q96Y52 type=del size=2064704 2024-03-04T17:50:51.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.621+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=7M53BG8LTGUZ type=del size=1278312 2024-03-04T17:50:51.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.625+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=RS3CXOFEPE2M type=del 2024-03-04T17:50:51.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.633+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~CMWLFQM8DCBN.0 size=1048576 2024-03-04T17:50:51.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.637+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~CMWLFQM8DCBN.1 size=242981 2024-03-04T17:50:51.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.637+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=CMWLFQM8DCBN tag=FB34QTG4EQ0A type=write size=2340133 2024-03-04T17:50:51.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.641+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=44T022921JWT type=del 2024-03-04T17:50:51.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.641+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=TFHATJCKUG1E type=write size=818133 2024-03-04T17:50:51.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.645+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~43WHX51OX4P9.0 size=67881 2024-03-04T17:50:51.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.653+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=43WHX51OX4P9 tag=787IG6V4GZ9L type=write size=1116457 2024-03-04T17:50:51.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.657+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=BFVJW2SC7AEV type=del size=1701659 2024-03-04T17:50:51.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.661+0000 7fda08af98c0 1 < completed write operation key=1OZKPHMPV3OQ tag=I84PW6I7I11K type=write size=105410 2024-03-04T17:50:51.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.665+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=IWYZBRLBRYG5 type=del size=1271641 2024-03-04T17:50:51.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.665+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=88VT5T5HOFMM type=del 2024-03-04T17:50:51.669 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.669+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=BQN5BW6JFQ5Q type=write size=2004 2024-03-04T17:50:51.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.673+0000 7fda08af98c0 1 < canceled part key=_multipart_VJVORUG95FLC.2~FQ0FAXDTPP2S.0 size=1048576 2024-03-04T17:50:51.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.673+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~FQ0FAXDTPP2S.1 size=1048576 2024-03-04T17:50:51.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.673+0000 7fda08af98c0 1 < canceled part key=_multipart_VJVORUG95FLC.2~FQ0FAXDTPP2S.2 size=697329 2024-03-04T17:50:51.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.681+0000 7fda08af98c0 1 < canceled multipart upload key=VJVORUG95FLC upload=FQ0FAXDTPP2S tag=DJE8B9KWPKST type=write size=3843057 2024-03-04T17:50:51.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.685+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=3C55P1D0O1GI type=del size=1419009 2024-03-04T17:50:51.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.685+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~IH8G200I8SMO.0 size=1048576 2024-03-04T17:50:51.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.685+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~IH8G200I8SMO.1 size=1040229 2024-03-04T17:50:51.690 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.693+0000 7fda08af98c0 1 < canceled multipart upload key=JA4JK99I0942 upload=IH8G200I8SMO tag=IMEFGPWC5RYS type=write size=3137381 2024-03-04T17:50:51.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.697+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=KP9IYIFE2RR3 type=del size=3636887 2024-03-04T17:50:51.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.701+0000 7fda08af98c0 1 < completed write operation key=HCKJZUMAK0Z0 tag=9YBPSNUK3RD4 type=write size=731313 2024-03-04T17:50:51.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.705+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=BL8KIN92RPVD type=del size=887619 2024-03-04T17:50:51.705 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.705+0000 7fda08af98c0 -1 < failed to complete operation key=X5MEJ7G8X9HE tag=NNFDTNPM2XFO type=del size=2913549: (22) Invalid argument 2024-03-04T17:50:51.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.713+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~J4I0RPP06RQ3.0 size=472781 2024-03-04T17:50:51.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.713+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=J4I0RPP06RQ3 tag=5MMQN6Q2GQRZ type=write size=1521357 2024-03-04T17:50:51.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.717+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=Z3TX0MVKCKN3 type=del 2024-03-04T17:50:51.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.721+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~ZI3VYJ5TZSFA.0 size=1048576 2024-03-04T17:50:51.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.725+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~ZI3VYJ5TZSFA.1 size=313541 2024-03-04T17:50:51.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.725+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=ZI3VYJ5TZSFA tag=Z2AQDDRFABYN type=write size=2410693 2024-03-04T17:50:51.728 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.729+0000 7fda08af98c0 1 < canceled operation key=HCKJZUMAK0Z0 tag=VCXA2F8Q35KS type=del size=604814 2024-03-04T17:50:51.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.733+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=618H13MDXI91 type=del size=2618807 2024-03-04T17:50:51.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.737+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=RT07BSQIIS6D type=del 2024-03-04T17:50:51.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.741+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=K2KBBYQMYUVW type=del size=81549 2024-03-04T17:50:51.743 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.745+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~5NZZNK5XCF1O.0 size=1048576 2024-03-04T17:50:51.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.749+0000 7fda08af98c0 1 < completed part key=_multipart_0IGSMFORCSL9.2~5NZZNK5XCF1O.1 size=318621 2024-03-04T17:50:51.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.753+0000 7fda08af98c0 1 < completed multipart upload key=0IGSMFORCSL9 upload=5NZZNK5XCF1O tag=0M33XEQHWAL5 type=write size=2415773 2024-03-04T17:50:51.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.757+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=ZSBCGS6CFQ8G type=write size=160181 2024-03-04T17:50:51.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.761+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~0I1M7JU3ZNKN.0 size=1048576 2024-03-04T17:50:51.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.765+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~0I1M7JU3ZNKN.1 size=1048576 2024-03-04T17:50:51.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.765+0000 7fda08af98c0 1 < canceled part key=_multipart_FMPZ2548LBBF.2~0I1M7JU3ZNKN.2 size=378278 2024-03-04T17:50:51.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.769+0000 7fda08af98c0 1 < canceled multipart upload key=FMPZ2548LBBF upload=0I1M7JU3ZNKN tag=TV3L7IORQAPM type=write size=3524006 2024-03-04T17:50:51.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.773+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=H38UVEQ7ITJM type=del size=18875 2024-03-04T17:50:51.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.777+0000 7fda08af98c0 1 < completed write operation key=Z3J7W9V8HQKH tag=4KIUMZ9I0NK0 type=write size=603901 2024-03-04T17:50:51.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.785+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~1902VJPCX7WJ.0 size=545623 2024-03-04T17:50:51.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.785+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=1902VJPCX7WJ tag=C8O7DGPDBYQQ type=write size=1594199 2024-03-04T17:50:51.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.789+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=7M53BG8LTGUZ type=del 2024-03-04T17:50:51.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.793+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=DRU06E02049A type=del size=2096659 2024-03-04T17:50:51.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.797+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~CMWLFQM8DCBN.0 size=1048576 2024-03-04T17:50:51.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.801+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~CMWLFQM8DCBN.1 size=242981 2024-03-04T17:50:51.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.805+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=CMWLFQM8DCBN tag=FB34QTG4EQ0A type=write size=2340133 2024-03-04T17:50:51.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.813+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~F9L9T7QO1EI3.0 size=624987 2024-03-04T17:50:51.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.813+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=F9L9T7QO1EI3 tag=BHZ1XV4XHRAO type=write size=1673563 2024-03-04T17:50:51.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.817+0000 7fda08af98c0 1 < canceled operation key=GDGTJZ21TG32 tag=TFHATJCKUG1E type=write size=818133 2024-03-04T17:50:51.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.821+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~8V2CIBNJH9J6.0 size=1048576 2024-03-04T17:50:51.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.825+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~8V2CIBNJH9J6.1 size=230662 2024-03-04T17:50:51.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.825+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=8V2CIBNJH9J6 tag=25A1SKSSMGPW type=write size=2327814 2024-03-04T17:50:51.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.829+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=BFVJW2SC7AEV type=del 2024-03-04T17:50:51.831 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.833+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=K2DS627FO10U type=del size=1794242 2024-03-04T17:50:51.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.837+0000 7fda08af98c0 1 < completed delete operation key=JA4JK99I0942 tag=IWYZBRLBRYG5 type=del 2024-03-04T17:50:51.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.841+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=9T9160ZRXYUG type=del size=1796332 2024-03-04T17:50:51.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.845+0000 7fda08af98c0 1 < canceled operation key=DJCCQ6S0E7UG tag=BQN5BW6JFQ5Q type=write size=2004 2024-03-04T17:50:51.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.849+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=QQSJ4F0EFJ6Q type=write size=356644 2024-03-04T17:50:51.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.853+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=3C55P1D0O1GI type=del 2024-03-04T17:50:51.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.857+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=DH23JCVUD8GT type=del size=689508 2024-03-04T17:50:51.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.861+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=KP9IYIFE2RR3 type=del 2024-03-04T17:50:51.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.865+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~HW4TQORTK3AC.0 size=1048576 2024-03-04T17:50:51.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.869+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~HW4TQORTK3AC.1 size=1048576 2024-03-04T17:50:51.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.877+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~HW4TQORTK3AC.2 size=749484 2024-03-04T17:50:51.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.877+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=HW4TQORTK3AC tag=O29ULJJQVPCI type=write size=3895212 2024-03-04T17:50:51.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.881+0000 7fda08af98c0 1 < canceled operation key=M5BMX7B7RZYL tag=BL8KIN92RPVD type=del size=887619 2024-03-04T17:50:51.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.881+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=W62BO4MZ8SZ9 type=del size=4078116 2024-03-04T17:50:51.885 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.889+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~J4I0RPP06RQ3.0 size=472781 2024-03-04T17:50:51.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.893+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=J4I0RPP06RQ3 tag=5MMQN6Q2GQRZ type=write size=1521357 2024-03-04T17:50:51.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.901+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~VP4M5V6WGTAJ.0 size=1048576 2024-03-04T17:50:51.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.901+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~VP4M5V6WGTAJ.1 size=75292 2024-03-04T17:50:51.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.901+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=VP4M5V6WGTAJ tag=U241R70296TN type=write size=2172444 2024-03-04T17:50:51.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.909+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~ZI3VYJ5TZSFA.0 size=1048576 2024-03-04T17:50:51.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.913+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~ZI3VYJ5TZSFA.1 size=313541 2024-03-04T17:50:51.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.917+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=ZI3VYJ5TZSFA tag=Z2AQDDRFABYN type=write size=2410693 2024-03-04T17:50:51.919 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.921+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=KWQWRMKEM4ZH type=write size=554962 2024-03-04T17:50:51.923 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.925+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=618H13MDXI91 type=del 2024-03-04T17:50:51.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.929+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=78C4F2OMCKKB type=del size=3383176 2024-03-04T17:50:51.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.937+0000 7fda08af98c0 1 < suggested remove operation key=L0FVKWD7EQRW tag=K2KBBYQMYUVW type=del 2024-03-04T17:50:51.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.941+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~EXCQGTGXX2D5.0 size=1048576 2024-03-04T17:50:51.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.945+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~EXCQGTGXX2D5.1 size=917724 2024-03-04T17:50:51.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.945+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=EXCQGTGXX2D5 tag=5ENX8W5I4Q80 type=write size=3014876 2024-03-04T17:50:51.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.949+0000 7fda08af98c0 -1 < failed to complete operation key=L0FVKWD7EQRW tag=ZSBCGS6CFQ8G type=write size=160181: (22) Invalid argument 2024-03-04T17:50:51.950 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.953+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=7II35WKIADLM type=del size=2361819 2024-03-04T17:50:51.954 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.957+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=H38UVEQ7ITJM type=del 2024-03-04T17:50:51.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.957+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=2ZTTR9BH76P1 type=del size=4076390 2024-03-04T17:50:51.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.961+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~1902VJPCX7WJ.0 size=545623 2024-03-04T17:50:51.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.969+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=1902VJPCX7WJ tag=C8O7DGPDBYQQ type=write size=1594199 2024-03-04T17:50:51.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.973+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~7084ZMNSDVI0.0 size=1048576 2024-03-04T17:50:51.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.977+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~7084ZMNSDVI0.1 size=536673 2024-03-04T17:50:51.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.977+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=7084ZMNSDVI0 tag=MGN3Y12GI1U5 type=write size=2633825 2024-03-04T17:50:51.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.981+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=DRU06E02049A type=del 2024-03-04T17:50:51.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.985+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=T7JMAR3VO9V1 type=write size=589712 2024-03-04T17:50:51.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.989+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~F9L9T7QO1EI3.0 size=624987 2024-03-04T17:50:51.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:51.997+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=F9L9T7QO1EI3 tag=BHZ1XV4XHRAO type=write size=1673563 2024-03-04T17:50:51.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.001+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=39D3YGY6WJFT type=write size=706542 2024-03-04T17:50:52.002 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.005+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~8V2CIBNJH9J6.0 size=1048576 2024-03-04T17:50:52.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.005+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~8V2CIBNJH9J6.1 size=230662 2024-03-04T17:50:52.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.013+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=8V2CIBNJH9J6 tag=25A1SKSSMGPW type=write size=2327814 2024-03-04T17:50:52.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.017+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=8LRZAR32PH36 type=del size=312342 2024-03-04T17:50:52.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.025+0000 7fda08af98c0 1 < suggested update operation key=DJCCQ6S0E7UG tag=K2DS627FO10U type=del size=1794242 2024-03-04T17:50:52.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.033+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~S66DHIZR6AWB.0 size=950631 2024-03-04T17:50:52.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.033+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=S66DHIZR6AWB tag=EHXQNL5851EE type=write size=1999207 2024-03-04T17:50:52.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.037+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=9T9160ZRXYUG type=del 2024-03-04T17:50:52.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.045+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~WAB3WR05HCAF.0 size=255811 2024-03-04T17:50:52.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.045+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=WAB3WR05HCAF tag=PV3WPMQIPSDF type=write size=1304387 2024-03-04T17:50:52.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.049+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=QQSJ4F0EFJ6Q type=write size=356644 2024-03-04T17:50:52.050 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.053+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=AOI078SJLKSX type=del size=3404099 2024-03-04T17:50:52.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.057+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=DH23JCVUD8GT type=del 2024-03-04T17:50:52.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.061+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~4VI6DLWWHDWO.0 size=1048576 2024-03-04T17:50:52.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.065+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~4VI6DLWWHDWO.1 size=1011510 2024-03-04T17:50:52.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.065+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=4VI6DLWWHDWO tag=UWV05WOO3WRW type=write size=3108662 2024-03-04T17:50:52.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.069+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~HW4TQORTK3AC.0 size=1048576 2024-03-04T17:50:52.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.073+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~HW4TQORTK3AC.1 size=1048576 2024-03-04T17:50:52.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.077+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~HW4TQORTK3AC.2 size=749484 2024-03-04T17:50:52.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.081+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=HW4TQORTK3AC tag=O29ULJJQVPCI type=write size=3895212 2024-03-04T17:50:52.084 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.085+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=9XBRNW4SGB3X type=del size=1752085 2024-03-04T17:50:52.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.089+0000 7fda08af98c0 -1 < failed to complete operation key=DJCCQ6S0E7UG tag=W62BO4MZ8SZ9 type=del size=4078116: (22) Invalid argument 2024-03-04T17:50:52.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.093+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=XCC1CGF0OU4V type=del size=1068278 2024-03-04T17:50:52.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.097+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~VP4M5V6WGTAJ.0 size=1048576 2024-03-04T17:50:52.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.097+0000 7fda08af98c0 1 < canceled part key=_multipart_RXVSQVVNXVF0.2~VP4M5V6WGTAJ.1 size=75292 2024-03-04T17:50:52.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.105+0000 7fda08af98c0 1 < canceled multipart upload key=RXVSQVVNXVF0 upload=VP4M5V6WGTAJ tag=U241R70296TN type=write size=2172444 2024-03-04T17:50:52.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.109+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=DY8I3EP119FP type=del size=990630 2024-03-04T17:50:52.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.109+0000 7fda08af98c0 1 < completed write operation key=JA4JK99I0942 tag=KWQWRMKEM4ZH type=write size=554962 2024-03-04T17:50:52.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.113+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=IZNNE1LUCHND type=del size=982654 2024-03-04T17:50:52.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.117+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=78C4F2OMCKKB type=del 2024-03-04T17:50:52.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.121+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=ZWS4R7YQK0RF type=del size=141771 2024-03-04T17:50:52.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.125+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~EXCQGTGXX2D5.0 size=1048576 2024-03-04T17:50:52.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.129+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~EXCQGTGXX2D5.1 size=917724 2024-03-04T17:50:52.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.137+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=EXCQGTGXX2D5 tag=5ENX8W5I4Q80 type=write size=3014876 2024-03-04T17:50:52.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.141+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=I631VBESBAN3 type=del size=4170766 2024-03-04T17:50:52.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.145+0000 7fda08af98c0 1 < canceled operation key=8YL7C9PI58TK tag=7II35WKIADLM type=del size=2361819 2024-03-04T17:50:52.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.145+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=TB2P5P5JJOGZ type=del size=1588253 2024-03-04T17:50:52.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.149+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=2ZTTR9BH76P1 type=del 2024-03-04T17:50:52.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.157+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8NYL70Y2JQT8.0 size=1048576 2024-03-04T17:50:52.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.161+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8NYL70Y2JQT8.1 size=1048576 2024-03-04T17:50:52.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.161+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8NYL70Y2JQT8.2 size=325096 2024-03-04T17:50:52.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.161+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=8NYL70Y2JQT8 tag=GUDQSINK99JX type=write size=3470824 2024-03-04T17:50:52.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.165+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~7084ZMNSDVI0.0 size=1048576 2024-03-04T17:50:52.168 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.169+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~7084ZMNSDVI0.1 size=536673 2024-03-04T17:50:52.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.177+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=7084ZMNSDVI0 tag=MGN3Y12GI1U5 type=write size=2633825 2024-03-04T17:50:52.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.181+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=S9SZP0LARM2Q type=del size=158506 2024-03-04T17:50:52.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.185+0000 7fda08af98c0 -1 < failed to complete operation key=DJCCQ6S0E7UG tag=T7JMAR3VO9V1 type=write size=589712: (22) Invalid argument 2024-03-04T17:50:52.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.189+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=8YGUTZMC8QP6 type=del size=1564687 2024-03-04T17:50:52.189 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.193+0000 7fda08af98c0 1 < completed write operation key=Z3J7W9V8HQKH tag=39D3YGY6WJFT type=write size=706542 2024-03-04T17:50:52.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.193+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=18QRA9CP0NN8 type=del size=2420894 2024-03-04T17:50:52.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.197+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=8LRZAR32PH36 type=del 2024-03-04T17:50:52.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.201+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=0OBEUAZUR8KH type=del size=3573134 2024-03-04T17:50:52.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.205+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~S66DHIZR6AWB.0 size=950631 2024-03-04T17:50:52.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.213+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=S66DHIZR6AWB tag=EHXQNL5851EE type=write size=1999207 2024-03-04T17:50:52.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.217+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=4Q61AWRT8R2K type=del size=611249 2024-03-04T17:50:52.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.221+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~WAB3WR05HCAF.0 size=255811 2024-03-04T17:50:52.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.229+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=WAB3WR05HCAF tag=PV3WPMQIPSDF type=write size=1304387 2024-03-04T17:50:52.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.233+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=Z874UU9JNTKI type=del size=1135114 2024-03-04T17:50:52.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.237+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=AOI078SJLKSX type=del 2024-03-04T17:50:52.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.241+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=X9XJFV0GQVMX type=del size=3752183 2024-03-04T17:50:52.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.245+0000 7fda08af98c0 1 < canceled part key=_multipart_XG5I3SDRE5JM.2~4VI6DLWWHDWO.0 size=1048576 2024-03-04T17:50:52.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.245+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~4VI6DLWWHDWO.1 size=1011510 2024-03-04T17:50:52.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.253+0000 7fda08af98c0 1 < canceled multipart upload key=XG5I3SDRE5JM upload=4VI6DLWWHDWO tag=UWV05WOO3WRW type=write size=3108662 2024-03-04T17:50:52.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.257+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=CC2BLYODX8EG type=del size=4148963 2024-03-04T17:50:52.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.261+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=9XBRNW4SGB3X type=del 2024-03-04T17:50:52.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.269+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~KWFH2OZFI6YS.0 size=661682 2024-03-04T17:50:52.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.269+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=KWFH2OZFI6YS tag=DEYQ4H2AQQ4U type=write size=1710258 2024-03-04T17:50:52.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.273+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=XCC1CGF0OU4V type=del 2024-03-04T17:50:52.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.281+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~WHUDQYPPAFKS.0 size=1048576 2024-03-04T17:50:52.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.281+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~WHUDQYPPAFKS.1 size=1048576 2024-03-04T17:50:52.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.285+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~WHUDQYPPAFKS.2 size=414957 2024-03-04T17:50:52.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.285+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=WHUDQYPPAFKS tag=JIXWR9YJAQQ6 type=write size=3560685 2024-03-04T17:50:52.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.293+0000 7fda08af98c0 1 < suggested update operation key=FNWA2N9U14HO tag=DY8I3EP119FP type=del size=990630 2024-03-04T17:50:52.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.297+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=JGABUJMR6730 type=del size=1903183 2024-03-04T17:50:52.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.305+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=IZNNE1LUCHND type=del 2024-03-04T17:50:52.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.309+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=F9BGXEU3CQMG type=write size=242894 2024-03-04T17:50:52.310 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.313+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=ZWS4R7YQK0RF type=del 2024-03-04T17:50:52.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.317+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=4FWLWNA46MX2 type=del size=411087 2024-03-04T17:50:52.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.321+0000 7fda08af98c0 1 < canceled operation key=C1GK00FM2ZYA tag=I631VBESBAN3 type=del size=4170766 2024-03-04T17:50:52.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.325+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=D0XW6738W63P type=del size=1616019 2024-03-04T17:50:52.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.329+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=TB2P5P5JJOGZ type=del 2024-03-04T17:50:52.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.337+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~96W2CUM82L1X.0 size=1048576 2024-03-04T17:50:52.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.337+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~96W2CUM82L1X.1 size=1048576 2024-03-04T17:50:52.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.341+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~96W2CUM82L1X.2 size=346830 2024-03-04T17:50:52.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.341+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=96W2CUM82L1X tag=H9TJCDLLT25L type=write size=3492558 2024-03-04T17:50:52.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.345+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~8NYL70Y2JQT8.0 size=1048576 2024-03-04T17:50:52.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.349+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~8NYL70Y2JQT8.1 size=1048576 2024-03-04T17:50:52.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.353+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~8NYL70Y2JQT8.2 size=325096 2024-03-04T17:50:52.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.357+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=8NYL70Y2JQT8 tag=GUDQSINK99JX type=write size=3470824 2024-03-04T17:50:52.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.361+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=FUFUTMTV5I3T type=write size=328394 2024-03-04T17:50:52.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.365+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=S9SZP0LARM2Q type=del 2024-03-04T17:50:52.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.369+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~59ZFV9TPQXAF.0 size=1048576 2024-03-04T17:50:52.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.373+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~59ZFV9TPQXAF.1 size=600596 2024-03-04T17:50:52.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.373+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=59ZFV9TPQXAF tag=RVOCE0RLGMK9 type=write size=2697748 2024-03-04T17:50:52.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.377+0000 7fda08af98c0 1 < canceled operation key=BYQFZSJ3WH07 tag=8YGUTZMC8QP6 type=del size=1564687 2024-03-04T17:50:52.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.381+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=D1DRDWB7RYFZ type=write size=497439 2024-03-04T17:50:52.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.385+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=18QRA9CP0NN8 type=del 2024-03-04T17:50:52.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.389+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=O7DUV5SJIA0W type=del size=856038 2024-03-04T17:50:52.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.393+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=0OBEUAZUR8KH type=del 2024-03-04T17:50:52.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.401+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~4XCHTHAIJU0K.0 size=1048576 2024-03-04T17:50:52.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.405+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~4XCHTHAIJU0K.1 size=236808 2024-03-04T17:50:52.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.405+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=4XCHTHAIJU0K tag=YDW7AAZXYBPT type=write size=2333960 2024-03-04T17:50:52.410 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.413+0000 7fda08af98c0 1 < suggested update operation key=M5BMX7B7RZYL tag=4Q61AWRT8R2K type=del size=611249 2024-03-04T17:50:52.414 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.417+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=5IIXS1JL0B0I type=del size=1521573 2024-03-04T17:50:52.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.425+0000 7fda08af98c0 1 < suggested update operation key=0IGSMFORCSL9 tag=Z874UU9JNTKI type=del size=1135114 2024-03-04T17:50:52.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.429+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=LB6K68K2RUK0 type=del size=2211872 2024-03-04T17:50:52.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.433+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=X9XJFV0GQVMX type=del 2024-03-04T17:50:52.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.437+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=D4OPIVOHIFBJ type=del size=4150612 2024-03-04T17:50:52.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.441+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=CC2BLYODX8EG type=del 2024-03-04T17:50:52.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.445+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=0I5OLOQWEWTI type=del size=1486249 2024-03-04T17:50:52.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.449+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~KWFH2OZFI6YS.0 size=661682 2024-03-04T17:50:52.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.457+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=KWFH2OZFI6YS tag=DEYQ4H2AQQ4U type=write size=1710258 2024-03-04T17:50:52.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.461+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=W3G9RVAM5ZOJ type=del size=518128 2024-03-04T17:50:52.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.465+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~WHUDQYPPAFKS.0 size=1048576 2024-03-04T17:50:52.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.465+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~WHUDQYPPAFKS.1 size=1048576 2024-03-04T17:50:52.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.469+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~WHUDQYPPAFKS.2 size=414957 2024-03-04T17:50:52.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.477+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=WHUDQYPPAFKS tag=JIXWR9YJAQQ6 type=write size=3560685 2024-03-04T17:50:52.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.481+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~P02N5YUT7AHD.0 size=1048576 2024-03-04T17:50:52.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.485+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~P02N5YUT7AHD.1 size=749186 2024-03-04T17:50:52.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.485+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=P02N5YUT7AHD tag=UZETZ5D5C8LM type=write size=2846338 2024-03-04T17:50:52.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.493+0000 7fda08af98c0 1 < suggested remove operation key=BYQFZSJ3WH07 tag=JGABUJMR6730 type=del 2024-03-04T17:50:52.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.501+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~OFPB9L3D4YX0.0 size=734534 2024-03-04T17:50:52.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.501+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=OFPB9L3D4YX0 tag=31VM0WJPX6MU type=write size=1783110 2024-03-04T17:50:52.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.509+0000 7fda08af98c0 1 < suggested update operation key=GDGTJZ21TG32 tag=F9BGXEU3CQMG type=write size=242894 2024-03-04T17:50:52.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.513+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=B61EXFU2OJMV type=del size=480968 2024-03-04T17:50:52.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.517+0000 7fda08af98c0 1 < completed delete operation key=ITIVF3UAT9NK tag=4FWLWNA46MX2 type=del 2024-03-04T17:50:52.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.525+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~U7X3J9KEPJST.0 size=1048576 2024-03-04T17:50:52.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.525+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~U7X3J9KEPJST.1 size=1048576 2024-03-04T17:50:52.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.529+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~U7X3J9KEPJST.2 size=923146 2024-03-04T17:50:52.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.529+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=U7X3J9KEPJST tag=7B32QO3T5Z98 type=write size=4068874 2024-03-04T17:50:52.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.533+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=D0XW6738W63P type=del 2024-03-04T17:50:52.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.541+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~707QKG8HHLKO.0 size=1048576 2024-03-04T17:50:52.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.545+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~707QKG8HHLKO.1 size=1048576 2024-03-04T17:50:52.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.545+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~707QKG8HHLKO.2 size=569186 2024-03-04T17:50:52.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.545+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=707QKG8HHLKO tag=OT6M1S9I4VYL type=write size=3714914 2024-03-04T17:50:52.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.549+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~96W2CUM82L1X.0 size=1048576 2024-03-04T17:50:52.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.553+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~96W2CUM82L1X.1 size=1048576 2024-03-04T17:50:52.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.557+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~96W2CUM82L1X.2 size=346830 2024-03-04T17:50:52.561 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.561+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=96W2CUM82L1X tag=H9TJCDLLT25L type=write size=3492558 2024-03-04T17:50:52.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.565+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=GD7ZWZS5KGQA type=write size=261225 2024-03-04T17:50:52.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.569+0000 7fda08af98c0 1 < completed write operation key=ITIVF3UAT9NK tag=FUFUTMTV5I3T type=write size=328394 2024-03-04T17:50:52.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.573+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=V9KMH8PD584V type=del size=3057833 2024-03-04T17:50:52.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.577+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~59ZFV9TPQXAF.0 size=1048576 2024-03-04T17:50:52.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.581+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~59ZFV9TPQXAF.1 size=600596 2024-03-04T17:50:52.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.585+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=59ZFV9TPQXAF tag=RVOCE0RLGMK9 type=write size=2697748 2024-03-04T17:50:52.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.597+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~DNNHPEOCEWAN.0 size=1048576 2024-03-04T17:50:52.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.601+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~DNNHPEOCEWAN.1 size=1018056 2024-03-04T17:50:52.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.601+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=DNNHPEOCEWAN tag=DS8D5KIE2GUT type=write size=3115208 2024-03-04T17:50:52.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.605+0000 7fda08af98c0 1 < completed write operation key=C4B45GL40IMZ tag=D1DRDWB7RYFZ type=write size=497439 2024-03-04T17:50:52.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.609+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=O9TCPSAU36OP type=del size=1818385 2024-03-04T17:50:52.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.617+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=O7DUV5SJIA0W type=del 2024-03-04T17:50:52.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.621+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=64TDXU05MQ2Y type=del size=948497 2024-03-04T17:50:52.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.625+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~4XCHTHAIJU0K.0 size=1048576 2024-03-04T17:50:52.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.625+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~4XCHTHAIJU0K.1 size=236808 2024-03-04T17:50:52.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.633+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=4XCHTHAIJU0K tag=YDW7AAZXYBPT type=write size=2333960 2024-03-04T17:50:52.635 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.637+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=2SU3H5HFW1PO type=write size=972195 2024-03-04T17:50:52.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.641+0000 7fda08af98c0 1 < canceled operation key=X5MEJ7G8X9HE tag=5IIXS1JL0B0I type=del size=1521573 2024-03-04T17:50:52.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.645+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=4DG33LFONK5I type=write size=244302 2024-03-04T17:50:52.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.649+0000 7fda08af98c0 1 < canceled operation key=X5MEJ7G8X9HE tag=LB6K68K2RUK0 type=del size=2211872 2024-03-04T17:50:52.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.653+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=7Y3MN9GBQ8HP type=del size=708510 2024-03-04T17:50:52.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.657+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=D4OPIVOHIFBJ type=del 2024-03-04T17:50:52.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.661+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=0COFXXH9RBVJ type=del size=989271 2024-03-04T17:50:52.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.665+0000 7fda08af98c0 -1 < failed to complete operation key=GDGTJZ21TG32 tag=0I5OLOQWEWTI type=del size=1486249: (22) Invalid argument 2024-03-04T17:50:52.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.673+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~9Y8LVH0WE02T.0 size=1048576 2024-03-04T17:50:52.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.677+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~9Y8LVH0WE02T.1 size=1048576 2024-03-04T17:50:52.677 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.677+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~9Y8LVH0WE02T.2 size=946854 2024-03-04T17:50:52.677 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.677+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=9Y8LVH0WE02T tag=K9ZR7UV2S39U type=write size=4092582 2024-03-04T17:50:52.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.681+0000 7fda08af98c0 -1 < failed to complete operation key=BYQFZSJ3WH07 tag=W3G9RVAM5ZOJ type=del size=518128: (22) Invalid argument 2024-03-04T17:50:52.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.689+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~DGXM6DZ8LVE3.0 size=151029 2024-03-04T17:50:52.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.689+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=DGXM6DZ8LVE3 tag=AI15ZYRSFIOQ type=write size=1199605 2024-03-04T17:50:52.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.693+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~P02N5YUT7AHD.0 size=1048576 2024-03-04T17:50:52.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.697+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~P02N5YUT7AHD.1 size=749186 2024-03-04T17:50:52.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.705+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=P02N5YUT7AHD tag=UZETZ5D5C8LM type=write size=2846338 2024-03-04T17:50:52.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.713+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~PIZ33CJZZXP8.0 size=1048576 2024-03-04T17:50:52.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.713+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~PIZ33CJZZXP8.1 size=229902 2024-03-04T17:50:52.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.713+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=PIZ33CJZZXP8 tag=D9C89AMQMM0B type=write size=2327054 2024-03-04T17:50:52.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.717+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~OFPB9L3D4YX0.0 size=734534 2024-03-04T17:50:52.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.725+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=OFPB9L3D4YX0 tag=31VM0WJPX6MU type=write size=1783110 2024-03-04T17:50:52.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.733+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~J52NMEG1O0UK.0 size=1048576 2024-03-04T17:50:52.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.737+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~J52NMEG1O0UK.1 size=1000574 2024-03-04T17:50:52.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.737+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=J52NMEG1O0UK tag=3VMAB7PUFQDI type=write size=3097726 2024-03-04T17:50:52.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.737+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=B61EXFU2OJMV type=del 2024-03-04T17:50:52.745 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.745+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~13MKBUF8JPJQ.0 size=559509 2024-03-04T17:50:52.745 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.745+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=13MKBUF8JPJQ tag=W295NW2EA7E5 type=write size=1608085 2024-03-04T17:50:52.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.753+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~U7X3J9KEPJST.0 size=1048576 2024-03-04T17:50:52.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.753+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~U7X3J9KEPJST.1 size=1048576 2024-03-04T17:50:52.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.757+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~U7X3J9KEPJST.2 size=923146 2024-03-04T17:50:52.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.765+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=U7X3J9KEPJST tag=7B32QO3T5Z98 type=write size=4068874 2024-03-04T17:50:52.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.769+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=10FCPVQJ26N4 type=del size=1718729 2024-03-04T17:50:52.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.773+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~707QKG8HHLKO.0 size=1048576 2024-03-04T17:50:52.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.777+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~707QKG8HHLKO.1 size=1048576 2024-03-04T17:50:52.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.781+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~707QKG8HHLKO.2 size=569186 2024-03-04T17:50:52.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.785+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=707QKG8HHLKO tag=OT6M1S9I4VYL type=write size=3714914 2024-03-04T17:50:52.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.793+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~J53XMQHHADKG.0 size=1048576 2024-03-04T17:50:52.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.797+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~J53XMQHHADKG.1 size=190751 2024-03-04T17:50:52.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.797+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=J53XMQHHADKG tag=5FQANFPLLAB5 type=write size=2287903 2024-03-04T17:50:52.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.801+0000 7fda08af98c0 1 < completed write operation key=AO3QS6CM7I5D tag=GD7ZWZS5KGQA type=write size=261225 2024-03-04T17:50:52.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.805+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=6FRQKMS5YACF type=del size=769537 2024-03-04T17:50:52.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.809+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=V9KMH8PD584V type=del 2024-03-04T17:50:52.812 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.813+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=ADD6AF78FDDG type=del size=2355432 2024-03-04T17:50:52.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.813+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~DNNHPEOCEWAN.0 size=1048576 2024-03-04T17:50:52.816 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.817+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~DNNHPEOCEWAN.1 size=1018056 2024-03-04T17:50:52.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.825+0000 7fda08af98c0 1 < canceled multipart upload key=JA4JK99I0942 upload=DNNHPEOCEWAN tag=DS8D5KIE2GUT type=write size=3115208 2024-03-04T17:50:52.830 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.833+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~K72ISR0FH99Z.0 size=1048576 2024-03-04T17:50:52.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.837+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~K72ISR0FH99Z.1 size=286457 2024-03-04T17:50:52.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.837+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=K72ISR0FH99Z tag=8D2V9TUJSTYN type=write size=2383609 2024-03-04T17:50:52.837 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.841+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=O9TCPSAU36OP type=del 2024-03-04T17:50:52.841 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.845+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=3MS8PO7KMKRK type=write size=472603 2024-03-04T17:50:52.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.849+0000 7fda08af98c0 1 < canceled operation key=RXVSQVVNXVF0 tag=64TDXU05MQ2Y type=del size=948497 2024-03-04T17:50:52.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.849+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=UUUQG11P6JM0 type=write size=136648 2024-03-04T17:50:52.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.857+0000 7fda08af98c0 1 < completed write operation key=DJCCQ6S0E7UG tag=2SU3H5HFW1PO type=write size=972195 2024-03-04T17:50:52.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.861+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=RGKE6C0XNDYL type=del size=2945934 2024-03-04T17:50:52.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.865+0000 7fda08af98c0 1 < completed write operation key=V1LDB2HXZ7EF tag=4DG33LFONK5I type=write size=244302 2024-03-04T17:50:52.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.865+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=7OAG3CMOSQMV type=write size=1020112 2024-03-04T17:50:52.869 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.869+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=7Y3MN9GBQ8HP type=del 2024-03-04T17:50:52.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.873+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=60LPT0K018E7 type=del size=1195387 2024-03-04T17:50:52.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.877+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=0COFXXH9RBVJ type=del 2024-03-04T17:50:52.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.885+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~599IACN6C2TQ.0 size=1048576 2024-03-04T17:50:52.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.889+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~599IACN6C2TQ.1 size=834425 2024-03-04T17:50:52.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.889+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=599IACN6C2TQ tag=IUXOJS5GWWEI type=write size=2931577 2024-03-04T17:50:52.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.893+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~9Y8LVH0WE02T.0 size=1048576 2024-03-04T17:50:52.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.897+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~9Y8LVH0WE02T.1 size=1048576 2024-03-04T17:50:52.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.897+0000 7fda08af98c0 1 < canceled part key=_multipart_GDGTJZ21TG32.2~9Y8LVH0WE02T.2 size=946854 2024-03-04T17:50:52.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.905+0000 7fda08af98c0 1 < canceled multipart upload key=GDGTJZ21TG32 upload=9Y8LVH0WE02T tag=K9ZR7UV2S39U type=write size=4092582 2024-03-04T17:50:52.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.909+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=HT66MRTTWL20 type=del size=1368488 2024-03-04T17:50:52.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.913+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~DGXM6DZ8LVE3.0 size=151029 2024-03-04T17:50:52.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.917+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=DGXM6DZ8LVE3 tag=AI15ZYRSFIOQ type=write size=1199605 2024-03-04T17:50:52.921 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.925+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=05EWKX3HG2GD type=del size=2446637 2024-03-04T17:50:52.925 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.929+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~PIZ33CJZZXP8.0 size=1048576 2024-03-04T17:50:52.928 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.929+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~PIZ33CJZZXP8.1 size=229902 2024-03-04T17:50:52.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.937+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=PIZ33CJZZXP8 tag=D9C89AMQMM0B type=write size=2327054 2024-03-04T17:50:52.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.945+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~WU11B40KYQPR.0 size=1048576 2024-03-04T17:50:52.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.945+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~WU11B40KYQPR.1 size=1048576 2024-03-04T17:50:52.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.949+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~WU11B40KYQPR.2 size=422213 2024-03-04T17:50:52.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.949+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=WU11B40KYQPR tag=BIEKIB9GWNIT type=write size=3567941 2024-03-04T17:50:52.953 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.953+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~J52NMEG1O0UK.0 size=1048576 2024-03-04T17:50:52.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.957+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~J52NMEG1O0UK.1 size=1000574 2024-03-04T17:50:52.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.969+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=J52NMEG1O0UK tag=3VMAB7PUFQDI type=write size=3097726 2024-03-04T17:50:52.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.977+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~4RC2KOC0PFO3.0 size=157597 2024-03-04T17:50:52.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.977+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=4RC2KOC0PFO3 tag=1HNM1QJ2OFGT type=write size=1206173 2024-03-04T17:50:52.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.981+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~13MKBUF8JPJQ.0 size=559509 2024-03-04T17:50:52.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.985+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=13MKBUF8JPJQ tag=W295NW2EA7E5 type=write size=1608085 2024-03-04T17:50:52.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.989+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=6O6FS1SS2EMT type=write size=906623 2024-03-04T17:50:52.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.993+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=10FCPVQJ26N4 type=del 2024-03-04T17:50:52.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:52.997+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=14RMBCOVQKN6 type=write size=1030044 2024-03-04T17:50:52.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.001+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~J53XMQHHADKG.0 size=1048576 2024-03-04T17:50:53.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.005+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~J53XMQHHADKG.1 size=190751 2024-03-04T17:50:53.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.009+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=J53XMQHHADKG tag=5FQANFPLLAB5 type=write size=2287903 2024-03-04T17:50:53.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.013+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=3H125LHNCOO9 type=del size=2054731 2024-03-04T17:50:53.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.017+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=6FRQKMS5YACF type=del 2024-03-04T17:50:53.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.025+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~DFGOS470X1TL.0 size=1048576 2024-03-04T17:50:53.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.029+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~DFGOS470X1TL.1 size=961442 2024-03-04T17:50:53.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.029+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=DFGOS470X1TL tag=PHTHX4EXRFIC type=write size=3058594 2024-03-04T17:50:53.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.037+0000 7fda08af98c0 1 < suggested update operation key=TOFWDFH0HL3Q tag=ADD6AF78FDDG type=del size=2355432 2024-03-04T17:50:53.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.041+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~XE1MAUJPGQZW.0 size=1048576 2024-03-04T17:50:53.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.045+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~XE1MAUJPGQZW.1 size=1048576 2024-03-04T17:50:53.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.049+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~XE1MAUJPGQZW.2 size=603995 2024-03-04T17:50:53.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.049+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=XE1MAUJPGQZW tag=9PWGBWIGNNER type=write size=3749723 2024-03-04T17:50:53.051 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.053+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~K72ISR0FH99Z.0 size=1048576 2024-03-04T17:50:53.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.057+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~K72ISR0FH99Z.1 size=286457 2024-03-04T17:50:53.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.061+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=K72ISR0FH99Z tag=8D2V9TUJSTYN type=write size=2383609 2024-03-04T17:50:53.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.065+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=KK1UX890099A type=del size=1272936 2024-03-04T17:50:53.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.069+0000 7fda08af98c0 1 < completed write operation key=XG2XYBCMUNY5 tag=3MS8PO7KMKRK type=write size=472603 2024-03-04T17:50:53.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.073+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=FCU0OCU6DDXX type=write size=583208 2024-03-04T17:50:53.076 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.077+0000 7fda08af98c0 1 < completed write operation key=HCKJZUMAK0Z0 tag=UUUQG11P6JM0 type=write size=136648 2024-03-04T17:50:53.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.085+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~5U1CTHGOFIP5.0 size=1048576 2024-03-04T17:50:53.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.089+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~5U1CTHGOFIP5.1 size=67284 2024-03-04T17:50:53.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.089+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=5U1CTHGOFIP5 tag=I4CJPRXCZF7Y type=write size=2164436 2024-03-04T17:50:53.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.097+0000 7fda08af98c0 1 < suggested update operation key=2PA8JDZUOLO5 tag=RGKE6C0XNDYL type=del size=2945934 2024-03-04T17:50:53.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.101+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=WP947WDHYHDS type=del size=287190 2024-03-04T17:50:53.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.105+0000 7fda08af98c0 1 < completed write operation key=VJVORUG95FLC tag=7OAG3CMOSQMV type=write size=1020112 2024-03-04T17:50:53.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.109+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=0XLOYENBXCDE type=del size=3003089 2024-03-04T17:50:53.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.117+0000 7fda08af98c0 1 < suggested update operation key=750NSVFBS03A tag=60LPT0K018E7 type=del size=1195387 2024-03-04T17:50:53.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.121+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=ZTK82KT0YAMZ type=del size=49742 2024-03-04T17:50:53.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.125+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~599IACN6C2TQ.0 size=1048576 2024-03-04T17:50:53.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.129+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~599IACN6C2TQ.1 size=834425 2024-03-04T17:50:53.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.133+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=599IACN6C2TQ tag=IUXOJS5GWWEI type=write size=2931577 2024-03-04T17:50:53.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.141+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~ST8MTV4I5GEU.0 size=1048576 2024-03-04T17:50:53.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.145+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~ST8MTV4I5GEU.1 size=1048576 2024-03-04T17:50:53.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.149+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~ST8MTV4I5GEU.2 size=239720 2024-03-04T17:50:53.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.149+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=ST8MTV4I5GEU tag=J6N06IGNWX83 type=write size=3385448 2024-03-04T17:50:53.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.153+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=HT66MRTTWL20 type=del 2024-03-04T17:50:53.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.157+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=D23DIZA98U31 type=write size=12424 2024-03-04T17:50:53.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.161+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=05EWKX3HG2GD type=del 2024-03-04T17:50:53.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.165+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~QNZI5NK100I3.0 size=1048576 2024-03-04T17:50:53.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.169+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~QNZI5NK100I3.1 size=564824 2024-03-04T17:50:53.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.169+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=QNZI5NK100I3 tag=7FQVQUXF0D2H type=write size=2661976 2024-03-04T17:50:53.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.173+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~WU11B40KYQPR.0 size=1048576 2024-03-04T17:50:53.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.177+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~WU11B40KYQPR.1 size=1048576 2024-03-04T17:50:53.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.177+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~WU11B40KYQPR.2 size=422213 2024-03-04T17:50:53.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.185+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=WU11B40KYQPR tag=BIEKIB9GWNIT type=write size=3567941 2024-03-04T17:50:53.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.193+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~EMH9SLJID4TR.0 size=1048576 2024-03-04T17:50:53.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.197+0000 7fda08af98c0 1 > prepared part key=_multipart_HCKJZUMAK0Z0.2~EMH9SLJID4TR.1 size=170572 2024-03-04T17:50:53.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.197+0000 7fda08af98c0 1 > prepared multipart upload key=HCKJZUMAK0Z0 upload=EMH9SLJID4TR tag=1GJZXQX24Z76 type=write size=2267724 2024-03-04T17:50:53.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.197+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~4RC2KOC0PFO3.0 size=157597 2024-03-04T17:50:53.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.205+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=4RC2KOC0PFO3 tag=1HNM1QJ2OFGT type=write size=1206173 2024-03-04T17:50:53.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.213+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~0BVALUSS0UHY.0 size=319102 2024-03-04T17:50:53.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.213+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=0BVALUSS0UHY tag=TYX3YY0BKMQB type=write size=1367678 2024-03-04T17:50:53.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.213+0000 7fda08af98c0 1 < completed write operation key=HCA7QLGM2O4O tag=6O6FS1SS2EMT type=write size=906623 2024-03-04T17:50:53.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.221+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~FLIEV1JRJM5C.0 size=814153 2024-03-04T17:50:53.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.221+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=FLIEV1JRJM5C tag=ZGQ3M631HDAB type=write size=1862729 2024-03-04T17:50:53.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.225+0000 7fda08af98c0 1 < completed write operation key=IJSYAVR5SWPS tag=14RMBCOVQKN6 type=write size=1030044 2024-03-04T17:50:53.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.233+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~HDJJWPH1677L.0 size=1048576 2024-03-04T17:50:53.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.233+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~HDJJWPH1677L.1 size=1048576 2024-03-04T17:50:53.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.237+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~HDJJWPH1677L.2 size=600474 2024-03-04T17:50:53.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.237+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=HDJJWPH1677L tag=G6QLWJY7JFUW type=write size=3746202 2024-03-04T17:50:53.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.241+0000 7fda08af98c0 1 < canceled operation key=IJSYAVR5SWPS tag=3H125LHNCOO9 type=del size=2054731 2024-03-04T17:50:53.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.245+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=E30VGQD1CGF1 type=write size=511888 2024-03-04T17:50:53.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.249+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~DFGOS470X1TL.0 size=1048576 2024-03-04T17:50:53.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.253+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~DFGOS470X1TL.1 size=961442 2024-03-04T17:50:53.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.257+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=DFGOS470X1TL tag=PHTHX4EXRFIC type=write size=3058594 2024-03-04T17:50:53.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.261+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=H1TBZZ6J0ZHE type=del size=3671056 2024-03-04T17:50:53.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.265+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~XE1MAUJPGQZW.0 size=1048576 2024-03-04T17:50:53.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.269+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~XE1MAUJPGQZW.1 size=1048576 2024-03-04T17:50:53.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.269+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~XE1MAUJPGQZW.2 size=603995 2024-03-04T17:50:53.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.277+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=XE1MAUJPGQZW tag=9PWGBWIGNNER type=write size=3749723 2024-03-04T17:50:53.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.281+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=NZ8XKGK81ILA type=del size=2951759 2024-03-04T17:50:53.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.285+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=KK1UX890099A type=del 2024-03-04T17:50:53.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.289+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=W4YZLGEHC2LM type=del size=2814846 2024-03-04T17:50:53.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.293+0000 7fda08af98c0 1 < completed write operation key=TOFWDFH0HL3Q tag=FCU0OCU6DDXX type=write size=583208 2024-03-04T17:50:53.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.297+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~DKZ3E1VT6RF7.0 size=1048576 2024-03-04T17:50:53.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.301+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~DKZ3E1VT6RF7.1 size=727142 2024-03-04T17:50:53.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.301+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=DKZ3E1VT6RF7 tag=5Y3D2SDICFV0 type=write size=2824294 2024-03-04T17:50:53.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.305+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~5U1CTHGOFIP5.0 size=1048576 2024-03-04T17:50:53.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.309+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~5U1CTHGOFIP5.1 size=67284 2024-03-04T17:50:53.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.317+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=5U1CTHGOFIP5 tag=I4CJPRXCZF7Y type=write size=2164436 2024-03-04T17:50:53.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.325+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~I4FOT6SI7IFK.0 size=1048576 2024-03-04T17:50:53.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.329+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~I4FOT6SI7IFK.1 size=474486 2024-03-04T17:50:53.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.329+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=I4FOT6SI7IFK tag=WOU4F69BS1DG type=write size=2571638 2024-03-04T17:50:53.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.333+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=WP947WDHYHDS type=del 2024-03-04T17:50:53.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.337+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=Q3W30A4HF5VZ type=del size=922526 2024-03-04T17:50:53.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.341+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=0XLOYENBXCDE type=del 2024-03-04T17:50:53.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.345+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=AQ543FDG78QX type=del size=758548 2024-03-04T17:50:53.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.345+0000 7fda08af98c0 1 < completed delete operation key=V1LDB2HXZ7EF tag=ZTK82KT0YAMZ type=del 2024-03-04T17:50:53.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.349+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=UL9U83HCBRCC type=del size=3764585 2024-03-04T17:50:53.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.353+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~ST8MTV4I5GEU.0 size=1048576 2024-03-04T17:50:53.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.357+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~ST8MTV4I5GEU.1 size=1048576 2024-03-04T17:50:53.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.361+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~ST8MTV4I5GEU.2 size=239720 2024-03-04T17:50:53.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.365+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=ST8MTV4I5GEU tag=J6N06IGNWX83 type=write size=3385448 2024-03-04T17:50:53.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.369+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=3SV7D7BRYMRN type=del size=3242312 2024-03-04T17:50:53.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.377+0000 7fda08af98c0 1 < completed write operation key=GDGTJZ21TG32 tag=D23DIZA98U31 type=write size=12424 2024-03-04T17:50:53.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.385+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~2UK03QH2ND75.0 size=1048576 2024-03-04T17:50:53.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.385+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~2UK03QH2ND75.1 size=1048576 2024-03-04T17:50:53.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.389+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~2UK03QH2ND75.2 size=132641 2024-03-04T17:50:53.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.389+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=2UK03QH2ND75 tag=L06X4JCQNYHL type=write size=3278369 2024-03-04T17:50:53.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.389+0000 7fda08af98c0 1 < canceled part key=_multipart_C4B45GL40IMZ.2~QNZI5NK100I3.0 size=1048576 2024-03-04T17:50:53.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.393+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~QNZI5NK100I3.1 size=564824 2024-03-04T17:50:53.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.401+0000 7fda08af98c0 1 < canceled multipart upload key=C4B45GL40IMZ upload=QNZI5NK100I3 tag=7FQVQUXF0D2H type=write size=2661976 2024-03-04T17:50:53.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.405+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=996ULXYKEHJZ type=del size=1435380 2024-03-04T17:50:53.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.409+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~EMH9SLJID4TR.0 size=1048576 2024-03-04T17:50:53.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.413+0000 7fda08af98c0 1 < completed part key=_multipart_HCKJZUMAK0Z0.2~EMH9SLJID4TR.1 size=170572 2024-03-04T17:50:53.416 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.417+0000 7fda08af98c0 1 < completed multipart upload key=HCKJZUMAK0Z0 upload=EMH9SLJID4TR tag=1GJZXQX24Z76 type=write size=2267724 2024-03-04T17:50:53.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.421+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=45DI6J3D5AP8 type=del size=1517563 2024-03-04T17:50:53.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.425+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~0BVALUSS0UHY.0 size=319102 2024-03-04T17:50:53.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.433+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=0BVALUSS0UHY tag=TYX3YY0BKMQB type=write size=1367678 2024-03-04T17:50:53.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.437+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~8IYD4VW62U5R.0 size=1048576 2024-03-04T17:50:53.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.441+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~8IYD4VW62U5R.1 size=1048576 2024-03-04T17:50:53.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.445+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~8IYD4VW62U5R.2 size=797718 2024-03-04T17:50:53.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.445+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=8IYD4VW62U5R tag=2TED93PMT3O8 type=write size=3943446 2024-03-04T17:50:53.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.449+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~FLIEV1JRJM5C.0 size=814153 2024-03-04T17:50:53.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.453+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=FLIEV1JRJM5C tag=ZGQ3M631HDAB type=write size=1862729 2024-03-04T17:50:53.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.461+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~7Q6ZTO3QAETE.0 size=397706 2024-03-04T17:50:53.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.461+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=7Q6ZTO3QAETE tag=7HDK4Z39O9OX type=write size=1446282 2024-03-04T17:50:53.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.461+0000 7fda08af98c0 1 < canceled part key=_multipart_RXVSQVVNXVF0.2~HDJJWPH1677L.0 size=1048576 2024-03-04T17:50:53.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.465+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~HDJJWPH1677L.1 size=1048576 2024-03-04T17:50:53.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.469+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~HDJJWPH1677L.2 size=600474 2024-03-04T17:50:53.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.477+0000 7fda08af98c0 1 < canceled multipart upload key=RXVSQVVNXVF0 upload=HDJJWPH1677L tag=G6QLWJY7JFUW type=write size=3746202 2024-03-04T17:50:53.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.481+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=QY8F5YM9000X type=del size=3232329 2024-03-04T17:50:53.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.485+0000 7fda08af98c0 1 < completed write operation key=8YL7C9PI58TK tag=E30VGQD1CGF1 type=write size=511888 2024-03-04T17:50:53.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.489+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=EPC1J6CWYDWY type=write size=559040 2024-03-04T17:50:53.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.493+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=H1TBZZ6J0ZHE type=del 2024-03-04T17:50:53.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.497+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=QLN9RYXL1HLF type=del size=832256 2024-03-04T17:50:53.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.501+0000 7fda08af98c0 1 < completed delete operation key=GDGTJZ21TG32 tag=NZ8XKGK81ILA type=del 2024-03-04T17:50:53.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.505+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=FH2M9UU5LS0Q type=del size=3955490 2024-03-04T17:50:53.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.509+0000 7fda08af98c0 1 < canceled operation key=AO3QS6CM7I5D tag=W4YZLGEHC2LM type=del size=2814846 2024-03-04T17:50:53.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.513+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=IPBN111863H6 type=del size=1183506 2024-03-04T17:50:53.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.517+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~DKZ3E1VT6RF7.0 size=1048576 2024-03-04T17:50:53.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.517+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~DKZ3E1VT6RF7.1 size=727142 2024-03-04T17:50:53.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.525+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=DKZ3E1VT6RF7 tag=5Y3D2SDICFV0 type=write size=2824294 2024-03-04T17:50:53.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.533+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~VDHJ083MPUM7.0 size=1048576 2024-03-04T17:50:53.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.537+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~VDHJ083MPUM7.1 size=683111 2024-03-04T17:50:53.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.537+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=VDHJ083MPUM7 tag=71IZW1PJV7LS type=write size=2780263 2024-03-04T17:50:53.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.537+0000 7fda08af98c0 1 < canceled part key=_multipart_X5MEJ7G8X9HE.2~I4FOT6SI7IFK.0 size=1048576 2024-03-04T17:50:53.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.537+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~I4FOT6SI7IFK.1 size=474486 2024-03-04T17:50:53.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.545+0000 7fda08af98c0 1 < canceled multipart upload key=X5MEJ7G8X9HE upload=I4FOT6SI7IFK tag=WOU4F69BS1DG type=write size=2571638 2024-03-04T17:50:53.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.553+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8JK2WFTNJ7B3.0 size=1048576 2024-03-04T17:50:53.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.553+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8JK2WFTNJ7B3.1 size=1048576 2024-03-04T17:50:53.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.557+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~8JK2WFTNJ7B3.2 size=968310 2024-03-04T17:50:53.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.557+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=8JK2WFTNJ7B3 tag=GQX00DEAXVPE type=write size=4114038 2024-03-04T17:50:53.563 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.565+0000 7fda08af98c0 1 < suggested update operation key=ITIVF3UAT9NK tag=Q3W30A4HF5VZ type=del size=922526 2024-03-04T17:50:53.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.569+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=14HZ01KFULEV type=del size=2241634 2024-03-04T17:50:53.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.573+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=AQ543FDG78QX type=del 2024-03-04T17:50:53.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.577+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=EGOEZPRBJRI6 type=del size=2964228 2024-03-04T17:50:53.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.581+0000 7fda08af98c0 -1 < failed to complete operation key=ITIVF3UAT9NK tag=UL9U83HCBRCC type=del size=3764585: (22) Invalid argument 2024-03-04T17:50:53.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.585+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~PZ2J83U16MNX.0 size=661512 2024-03-04T17:50:53.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.585+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=PZ2J83U16MNX tag=50XGHBUSCNAV type=write size=1710088 2024-03-04T17:50:53.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.593+0000 7fda08af98c0 1 < suggested update operation key=3UWSL7NJLACY tag=3SV7D7BRYMRN type=del size=3242312 2024-03-04T17:50:53.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.597+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=49ZHFKVE7HXM type=del size=36515 2024-03-04T17:50:53.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.601+0000 7fda08af98c0 1 < canceled part key=_multipart_09KBCA2XURTF.2~2UK03QH2ND75.0 size=1048576 2024-03-04T17:50:53.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.601+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~2UK03QH2ND75.1 size=1048576 2024-03-04T17:50:53.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.605+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~2UK03QH2ND75.2 size=132641 2024-03-04T17:50:53.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.613+0000 7fda08af98c0 1 < canceled multipart upload key=09KBCA2XURTF upload=2UK03QH2ND75 tag=L06X4JCQNYHL type=write size=3278369 2024-03-04T17:50:53.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.621+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~KK4ZIAVSPJNO.0 size=1048576 2024-03-04T17:50:53.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.621+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~KK4ZIAVSPJNO.1 size=1048576 2024-03-04T17:50:53.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.625+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~KK4ZIAVSPJNO.2 size=932089 2024-03-04T17:50:53.624 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.625+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=KK4ZIAVSPJNO tag=29PCA6NZTFGM type=write size=4077817 2024-03-04T17:50:53.628 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.629+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=996ULXYKEHJZ type=del 2024-03-04T17:50:53.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.637+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~WQU2AN5GWJWZ.0 size=1048576 2024-03-04T17:50:53.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.641+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~WQU2AN5GWJWZ.1 size=246557 2024-03-04T17:50:53.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.641+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=WQU2AN5GWJWZ tag=NVAGEF2LUEN8 type=write size=2343709 2024-03-04T17:50:53.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.649+0000 7fda08af98c0 1 < suggested update operation key=750NSVFBS03A tag=45DI6J3D5AP8 type=del size=1517563 2024-03-04T17:50:53.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.657+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~322XYZ44W3XR.0 size=1048576 2024-03-04T17:50:53.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.661+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~322XYZ44W3XR.1 size=1048576 2024-03-04T17:50:53.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.661+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~322XYZ44W3XR.2 size=776956 2024-03-04T17:50:53.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.661+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=322XYZ44W3XR tag=0PCG4DXU3H7E type=write size=3922684 2024-03-04T17:50:53.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.665+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~8IYD4VW62U5R.0 size=1048576 2024-03-04T17:50:53.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.669+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~8IYD4VW62U5R.1 size=1048576 2024-03-04T17:50:53.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.677+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~8IYD4VW62U5R.2 size=797718 2024-03-04T17:50:53.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.681+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=8IYD4VW62U5R tag=2TED93PMT3O8 type=write size=3943446 2024-03-04T17:50:53.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.685+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=CF5Q3NKUWZ2Q type=del size=1516686 2024-03-04T17:50:53.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.689+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~7Q6ZTO3QAETE.0 size=397706 2024-03-04T17:50:53.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.697+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=7Q6ZTO3QAETE tag=7HDK4Z39O9OX type=write size=1446282 2024-03-04T17:50:53.697 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.701+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=P03YH4JBD045 type=del size=3136244 2024-03-04T17:50:53.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.701+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=QY8F5YM9000X type=del 2024-03-04T17:50:53.705 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.705+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=OYCWVZ2DDGOA type=del size=3020542 2024-03-04T17:50:53.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.709+0000 7fda08af98c0 1 < completed write operation key=JA4JK99I0942 tag=EPC1J6CWYDWY type=write size=559040 2024-03-04T17:50:53.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.717+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~C79AZ5QS0GF7.0 size=577232 2024-03-04T17:50:53.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.717+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=C79AZ5QS0GF7 tag=1JCUIWEGZE1I type=write size=1625808 2024-03-04T17:50:53.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.721+0000 7fda08af98c0 1 < canceled operation key=C1GK00FM2ZYA tag=QLN9RYXL1HLF type=del size=832256 2024-03-04T17:50:53.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.725+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=LZYYPQFDYLD1 type=write size=893096 2024-03-04T17:50:53.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.729+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=FH2M9UU5LS0Q type=del 2024-03-04T17:50:53.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.733+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~0GP4M5IFN7XZ.0 size=1048576 2024-03-04T17:50:53.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.737+0000 7fda08af98c0 1 > prepared part key=_multipart_TOFWDFH0HL3Q.2~0GP4M5IFN7XZ.1 size=1020809 2024-03-04T17:50:53.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.737+0000 7fda08af98c0 1 > prepared multipart upload key=TOFWDFH0HL3Q upload=0GP4M5IFN7XZ tag=ASUH8IANL6NW type=write size=3117961 2024-03-04T17:50:53.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.741+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=IPBN111863H6 type=del 2024-03-04T17:50:53.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.749+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~11PSZWCCIUMS.0 size=953063 2024-03-04T17:50:53.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.749+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=11PSZWCCIUMS tag=V483F52GE8OO type=write size=2001639 2024-03-04T17:50:53.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.753+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~VDHJ083MPUM7.0 size=1048576 2024-03-04T17:50:53.754 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.757+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~VDHJ083MPUM7.1 size=683111 2024-03-04T17:50:53.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.761+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=VDHJ083MPUM7 tag=71IZW1PJV7LS type=write size=2780263 2024-03-04T17:50:53.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.765+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=VBG7SQX04I7F type=del size=2426651 2024-03-04T17:50:53.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.769+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~8JK2WFTNJ7B3.0 size=1048576 2024-03-04T17:50:53.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.773+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~8JK2WFTNJ7B3.1 size=1048576 2024-03-04T17:50:53.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.777+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~8JK2WFTNJ7B3.2 size=968310 2024-03-04T17:50:53.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.781+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=8JK2WFTNJ7B3 tag=GQX00DEAXVPE type=write size=4114038 2024-03-04T17:50:53.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.789+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=8EA2PYPJ0X4R type=write size=140413 2024-03-04T17:50:53.789 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.793+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=14HZ01KFULEV type=del 2024-03-04T17:50:53.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.801+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~7TWXSRWMZSJV.0 size=621861 2024-03-04T17:50:53.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.801+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=7TWXSRWMZSJV tag=XURAKJGB3CUZ type=write size=1670437 2024-03-04T17:50:53.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.805+0000 7fda08af98c0 -1 < failed to complete operation key=3UWSL7NJLACY tag=EGOEZPRBJRI6 type=del size=2964228: (22) Invalid argument 2024-03-04T17:50:53.806 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.809+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=YWB7EZKZ8E31 type=del size=1072534 2024-03-04T17:50:53.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.813+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~PZ2J83U16MNX.0 size=661512 2024-03-04T17:50:53.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.817+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=PZ2J83U16MNX tag=50XGHBUSCNAV type=write size=1710088 2024-03-04T17:50:53.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.821+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=3RABEAJP6LCO type=del size=43456 2024-03-04T17:50:53.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.829+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=49ZHFKVE7HXM type=del 2024-03-04T17:50:53.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.833+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=NQQ9MF8FVXFC type=write size=635779 2024-03-04T17:50:53.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.837+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~KK4ZIAVSPJNO.0 size=1048576 2024-03-04T17:50:53.837 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.837+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~KK4ZIAVSPJNO.1 size=1048576 2024-03-04T17:50:53.840 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.841+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~KK4ZIAVSPJNO.2 size=932089 2024-03-04T17:50:53.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.849+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=KK4ZIAVSPJNO tag=29PCA6NZTFGM type=write size=4077817 2024-03-04T17:50:53.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.857+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~Y66B7GBH97S8.0 size=1048576 2024-03-04T17:50:53.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.857+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~Y66B7GBH97S8.1 size=401762 2024-03-04T17:50:53.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.857+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=Y66B7GBH97S8 tag=WAYF2SADHUQH type=write size=2498914 2024-03-04T17:50:53.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.861+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~WQU2AN5GWJWZ.0 size=1048576 2024-03-04T17:50:53.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.861+0000 7fda08af98c0 1 < canceled part key=_multipart_C4B45GL40IMZ.2~WQU2AN5GWJWZ.1 size=246557 2024-03-04T17:50:53.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.869+0000 7fda08af98c0 1 < canceled multipart upload key=C4B45GL40IMZ upload=WQU2AN5GWJWZ tag=NVAGEF2LUEN8 type=write size=2343709 2024-03-04T17:50:53.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.873+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=5BRSFIMKWDRC type=write size=108568 2024-03-04T17:50:53.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.877+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~322XYZ44W3XR.0 size=1048576 2024-03-04T17:50:53.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.881+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~322XYZ44W3XR.1 size=1048576 2024-03-04T17:50:53.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.885+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~322XYZ44W3XR.2 size=776956 2024-03-04T17:50:53.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.893+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=322XYZ44W3XR tag=0PCG4DXU3H7E type=write size=3922684 2024-03-04T17:50:53.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.897+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=4Z4BEBBMNV20 type=del size=2573430 2024-03-04T17:50:53.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.901+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=CF5Q3NKUWZ2Q type=del 2024-03-04T17:50:53.903 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.905+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=W3P4OW6VQX81 type=del size=1146041 2024-03-04T17:50:53.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.909+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=P03YH4JBD045 type=del 2024-03-04T17:50:53.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.913+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=HKBO7F72NL58 type=del size=1775604 2024-03-04T17:50:53.920 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.921+0000 7fda08af98c0 1 < suggested update operation key=IJSYAVR5SWPS tag=OYCWVZ2DDGOA type=del size=3020542 2024-03-04T17:50:53.923 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.925+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=8JF0ZB1HWPHD type=write size=252430 2024-03-04T17:50:53.927 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.929+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~C79AZ5QS0GF7.0 size=577232 2024-03-04T17:50:53.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.937+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=C79AZ5QS0GF7 tag=1JCUIWEGZE1I type=write size=1625808 2024-03-04T17:50:53.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.941+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=QNEDJANTKAXF type=del size=1560343 2024-03-04T17:50:53.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.949+0000 7fda08af98c0 1 < suggested remove operation key=FMPZ2548LBBF tag=LZYYPQFDYLD1 type=write 2024-03-04T17:50:53.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.953+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=7BVJHRE9QGVT type=write size=83560 2024-03-04T17:50:53.955 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.957+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~0GP4M5IFN7XZ.0 size=1048576 2024-03-04T17:50:53.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.961+0000 7fda08af98c0 1 < completed part key=_multipart_TOFWDFH0HL3Q.2~0GP4M5IFN7XZ.1 size=1020809 2024-03-04T17:50:53.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.969+0000 7fda08af98c0 1 < completed multipart upload key=TOFWDFH0HL3Q upload=0GP4M5IFN7XZ tag=ASUH8IANL6NW type=write size=3117961 2024-03-04T17:50:53.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.973+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=Q1GE92LRQJMX type=del size=1635376 2024-03-04T17:50:53.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.973+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~11PSZWCCIUMS.0 size=953063 2024-03-04T17:50:53.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.981+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=11PSZWCCIUMS tag=V483F52GE8OO type=write size=2001639 2024-03-04T17:50:53.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.985+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=B1KQ7D6I6XNJ type=del size=2439862 2024-03-04T17:50:53.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.989+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=VBG7SQX04I7F type=del 2024-03-04T17:50:53.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:53.993+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=6GLQF94X61EZ type=del size=3407940 2024-03-04T17:50:53.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.001+0000 7fda08af98c0 1 < suggested remove operation key=AO3QS6CM7I5D tag=8EA2PYPJ0X4R type=write 2024-03-04T17:50:54.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.009+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~U0DA18Z9SKG9.0 size=1048576 2024-03-04T17:50:54.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.009+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~U0DA18Z9SKG9.1 size=463622 2024-03-04T17:50:54.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.009+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=U0DA18Z9SKG9 tag=4BY00UQS32KH type=write size=2560774 2024-03-04T17:50:54.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.013+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~7TWXSRWMZSJV.0 size=621861 2024-03-04T17:50:54.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.021+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=7TWXSRWMZSJV tag=XURAKJGB3CUZ type=write size=1670437 2024-03-04T17:50:54.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.025+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=Y28X9R0AV4WR type=del size=3997931 2024-03-04T17:50:54.025 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.029+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=YWB7EZKZ8E31 type=del 2024-03-04T17:50:54.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.033+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~7HR2VD3GAXCD.0 size=476006 2024-03-04T17:50:54.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.033+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=7HR2VD3GAXCD tag=42Z6ZGOOO7S5 type=write size=1524582 2024-03-04T17:50:54.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.045+0000 7fda08af98c0 1 < suggested remove operation key=V1LDB2HXZ7EF tag=3RABEAJP6LCO type=del 2024-03-04T17:50:54.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.049+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=WAM12OXPVQ3Q type=write size=749372 2024-03-04T17:50:54.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.053+0000 7fda08af98c0 1 < completed write operation key=GDGTJZ21TG32 tag=NQQ9MF8FVXFC type=write size=635779 2024-03-04T17:50:54.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.061+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~1CYDQXWYHEF7.0 size=49699 2024-03-04T17:50:54.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.061+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=1CYDQXWYHEF7 tag=OA2SI2F62SW3 type=write size=1098275 2024-03-04T17:50:54.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.065+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~Y66B7GBH97S8.0 size=1048576 2024-03-04T17:50:54.067 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.069+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~Y66B7GBH97S8.1 size=401762 2024-03-04T17:50:54.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.077+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=Y66B7GBH97S8 tag=WAYF2SADHUQH type=write size=2498914 2024-03-04T17:50:54.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.081+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~MMJFMR5W8C4N.0 size=1025102 2024-03-04T17:50:54.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.085+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=MMJFMR5W8C4N tag=WLEL0U2MA3L0 type=write size=2073678 2024-03-04T17:50:54.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.093+0000 7fda08af98c0 1 < suggested remove operation key=C4B45GL40IMZ tag=5BRSFIMKWDRC type=write 2024-03-04T17:50:54.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.097+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~K4OJBQRFTKWW.0 size=361354 2024-03-04T17:50:54.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.097+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=K4OJBQRFTKWW tag=5Q2WDPQE5FVB type=write size=1409930 2024-03-04T17:50:54.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.101+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=4Z4BEBBMNV20 type=del 2024-03-04T17:50:54.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.109+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=DY9RWHMD65II type=del size=1058989 2024-03-04T17:50:54.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.117+0000 7fda08af98c0 1 < suggested update operation key=8YL7C9PI58TK tag=W3P4OW6VQX81 type=del size=1146041 2024-03-04T17:50:54.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.121+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=IHSGOJ66PIM0 type=del size=4049124 2024-03-04T17:50:54.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.125+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=HKBO7F72NL58 type=del 2024-03-04T17:50:54.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.129+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=4PUIL1IPKQ8B type=del size=1864565 2024-03-04T17:50:54.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.133+0000 7fda08af98c0 -1 < failed to complete operation key=V1LDB2HXZ7EF tag=8JF0ZB1HWPHD type=write size=252430: (22) Invalid argument 2024-03-04T17:50:54.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.137+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=HTHDK4HG60OD type=del size=1141907 2024-03-04T17:50:54.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.141+0000 7fda08af98c0 1 < completed delete operation key=VJVORUG95FLC tag=QNEDJANTKAXF type=del 2024-03-04T17:50:54.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.145+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=QM2Y4QCPY1EI type=write size=558385 2024-03-04T17:50:54.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.153+0000 7fda08af98c0 1 < suggested update operation key=1OZKPHMPV3OQ tag=7BVJHRE9QGVT type=write size=83560 2024-03-04T17:50:54.158 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.161+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~XHAHAUDMKS8S.0 size=1048576 2024-03-04T17:50:54.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.165+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~XHAHAUDMKS8S.1 size=1000016 2024-03-04T17:50:54.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.165+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=XHAHAUDMKS8S tag=ZY9XN139WJNX type=write size=3097168 2024-03-04T17:50:54.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.169+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=Q1GE92LRQJMX type=del 2024-03-04T17:50:54.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.173+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=65QD64RWYGV0 type=del size=1156483 2024-03-04T17:50:54.176 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.177+0000 7fda08af98c0 1 < canceled operation key=0IGSMFORCSL9 tag=B1KQ7D6I6XNJ type=del size=2439862 2024-03-04T17:50:54.181 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.181+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=4WDTL0863QU1 type=del size=224970 2024-03-04T17:50:54.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.189+0000 7fda08af98c0 1 < completed delete operation key=BYQFZSJ3WH07 tag=6GLQF94X61EZ type=del 2024-03-04T17:50:54.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.193+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~TI9N7VSVM8Q4.0 size=1048576 2024-03-04T17:50:54.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.197+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~TI9N7VSVM8Q4.1 size=16373 2024-03-04T17:50:54.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.197+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=TI9N7VSVM8Q4 tag=1VCE6K4WEGBK type=write size=2113525 2024-03-04T17:50:54.199 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.201+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~U0DA18Z9SKG9.0 size=1048576 2024-03-04T17:50:54.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.205+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~U0DA18Z9SKG9.1 size=463622 2024-03-04T17:50:54.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.209+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=U0DA18Z9SKG9 tag=4BY00UQS32KH type=write size=2560774 2024-03-04T17:50:54.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.213+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=GISAWXVVUILU type=del size=2708324 2024-03-04T17:50:54.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.217+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=Y28X9R0AV4WR type=del 2024-03-04T17:50:54.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.225+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~TLY235W4ME93.0 size=1048576 2024-03-04T17:50:54.225 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.225+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~TLY235W4ME93.1 size=1048576 2024-03-04T17:50:54.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.229+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~TLY235W4ME93.2 size=759877 2024-03-04T17:50:54.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.229+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=TLY235W4ME93 tag=PDOZZYIQMXUH type=write size=3905605 2024-03-04T17:50:54.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.233+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~7HR2VD3GAXCD.0 size=476006 2024-03-04T17:50:54.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.241+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=7HR2VD3GAXCD tag=42Z6ZGOOO7S5 type=write size=1524582 2024-03-04T17:50:54.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.245+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=39UOPCD01G69 type=write size=602244 2024-03-04T17:50:54.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.249+0000 7fda08af98c0 1 < completed write operation key=XG5I3SDRE5JM tag=WAM12OXPVQ3Q type=write size=749372 2024-03-04T17:50:54.254 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.257+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~C3X6F0TD8UJS.0 size=1048576 2024-03-04T17:50:54.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.257+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~C3X6F0TD8UJS.1 size=956147 2024-03-04T17:50:54.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.257+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=C3X6F0TD8UJS tag=HLNUY2VWWV4T type=write size=3053299 2024-03-04T17:50:54.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.261+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~1CYDQXWYHEF7.0 size=49699 2024-03-04T17:50:54.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.269+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=1CYDQXWYHEF7 tag=OA2SI2F62SW3 type=write size=1098275 2024-03-04T17:50:54.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.277+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~6FFC401ZDPY9.0 size=1048576 2024-03-04T17:50:54.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.277+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~6FFC401ZDPY9.1 size=638459 2024-03-04T17:50:54.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.277+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=6FFC401ZDPY9 tag=2R0CNQ42SLNY type=write size=2735611 2024-03-04T17:50:54.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.281+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~MMJFMR5W8C4N.0 size=1025102 2024-03-04T17:50:54.286 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.289+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=MMJFMR5W8C4N tag=WLEL0U2MA3L0 type=write size=2073678 2024-03-04T17:50:54.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.293+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~USKN6SFZBXYF.0 size=1048576 2024-03-04T17:50:54.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.297+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~USKN6SFZBXYF.1 size=995205 2024-03-04T17:50:54.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.297+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=USKN6SFZBXYF tag=OOYF9DNTCYBD type=write size=3092357 2024-03-04T17:50:54.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.301+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~K4OJBQRFTKWW.0 size=361354 2024-03-04T17:50:54.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.309+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=K4OJBQRFTKWW tag=5Q2WDPQE5FVB type=write size=1409930 2024-03-04T17:50:54.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.313+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=IWBE25YQ3RNH type=del size=2656561 2024-03-04T17:50:54.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.317+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=DY9RWHMD65II type=del 2024-03-04T17:50:54.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.321+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=7EBLI3FQ30C0 type=del size=1949044 2024-03-04T17:50:54.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.325+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=IHSGOJ66PIM0 type=del 2024-03-04T17:50:54.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.329+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=OGLFCBBH4WI2 type=del size=2827160 2024-03-04T17:50:54.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.333+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=4PUIL1IPKQ8B type=del 2024-03-04T17:50:54.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.341+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~CQFTIB1FA0UH.0 size=1048576 2024-03-04T17:50:54.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.345+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~CQFTIB1FA0UH.1 size=1048576 2024-03-04T17:50:54.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.345+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~CQFTIB1FA0UH.2 size=308965 2024-03-04T17:50:54.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.345+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=CQFTIB1FA0UH tag=YQW038IQY8EJ type=write size=3454693 2024-03-04T17:50:54.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.353+0000 7fda08af98c0 1 < suggested remove operation key=L0FVKWD7EQRW tag=HTHDK4HG60OD type=del 2024-03-04T17:50:54.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.357+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=156AC5AIOMOX type=del size=1392265 2024-03-04T17:50:54.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.361+0000 7fda08af98c0 1 < canceled operation key=Z3J7W9V8HQKH tag=QM2Y4QCPY1EI type=write size=558385 2024-03-04T17:50:54.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.365+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=HT9OG0ARB7L5 type=del size=3480150 2024-03-04T17:50:54.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.365+0000 7fda08af98c0 1 < canceled part key=_multipart_XG5I3SDRE5JM.2~XHAHAUDMKS8S.0 size=1048576 2024-03-04T17:50:54.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.369+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~XHAHAUDMKS8S.1 size=1000016 2024-03-04T17:50:54.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.373+0000 7fda08af98c0 1 < canceled multipart upload key=XG5I3SDRE5JM upload=XHAHAUDMKS8S tag=ZY9XN139WJNX type=write size=3097168 2024-03-04T17:50:54.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.381+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~55D2L02EU202.0 size=1048576 2024-03-04T17:50:54.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.385+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~55D2L02EU202.1 size=1048576 2024-03-04T17:50:54.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.389+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~55D2L02EU202.2 size=635743 2024-03-04T17:50:54.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.389+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=55D2L02EU202 tag=P251SK06IHFZ type=write size=3781471 2024-03-04T17:50:54.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.393+0000 7fda08af98c0 1 < suggested update operation key=JA4JK99I0942 tag=65QD64RWYGV0 type=del size=1156483 2024-03-04T17:50:54.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.401+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~TVU59EYZVTY7.0 size=1048576 2024-03-04T17:50:54.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.405+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~TVU59EYZVTY7.1 size=1044268 2024-03-04T17:50:54.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.405+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=TVU59EYZVTY7 tag=PQCUDS0KH5AL type=write size=3141420 2024-03-04T17:50:54.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.409+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=4WDTL0863QU1 type=del 2024-03-04T17:50:54.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.421+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~9U9R29E8OZM0.0 size=1048576 2024-03-04T17:50:54.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.425+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~9U9R29E8OZM0.1 size=56893 2024-03-04T17:50:54.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.425+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=9U9R29E8OZM0 tag=N7JLRMNLH8JZ type=write size=2154045 2024-03-04T17:50:54.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.429+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~TI9N7VSVM8Q4.0 size=1048576 2024-03-04T17:50:54.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.433+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~TI9N7VSVM8Q4.1 size=16373 2024-03-04T17:50:54.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.437+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=TI9N7VSVM8Q4 tag=1VCE6K4WEGBK type=write size=2113525 2024-03-04T17:50:54.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.445+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~20THJ1ANX7RQ.0 size=1048576 2024-03-04T17:50:54.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.449+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~20THJ1ANX7RQ.1 size=1048576 2024-03-04T17:50:54.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.453+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~20THJ1ANX7RQ.2 size=410970 2024-03-04T17:50:54.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.453+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=20THJ1ANX7RQ tag=LIP9G47C22QN type=write size=3556698 2024-03-04T17:50:54.458 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.461+0000 7fda08af98c0 1 < suggested remove operation key=1OZKPHMPV3OQ tag=GISAWXVVUILU type=del 2024-03-04T17:50:54.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.469+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~QTG25IO0OR7W.0 size=993333 2024-03-04T17:50:54.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.469+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=QTG25IO0OR7W tag=M6XWEHWB4EKK type=write size=2041909 2024-03-04T17:50:54.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.473+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~TLY235W4ME93.0 size=1048576 2024-03-04T17:50:54.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.473+0000 7fda08af98c0 1 < canceled part key=_multipart_FNWA2N9U14HO.2~TLY235W4ME93.1 size=1048576 2024-03-04T17:50:54.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.477+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~TLY235W4ME93.2 size=759877 2024-03-04T17:50:54.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.481+0000 7fda08af98c0 1 < canceled multipart upload key=FNWA2N9U14HO upload=TLY235W4ME93 tag=PDOZZYIQMXUH type=write size=3905605 2024-03-04T17:50:54.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.485+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=R06TNDNSWQBE type=del size=3291488 2024-03-04T17:50:54.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.489+0000 7fda08af98c0 1 < completed write operation key=VJVORUG95FLC tag=39UOPCD01G69 type=write size=602244 2024-03-04T17:50:54.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.493+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=NPN28S1VPAEN type=del size=3059357 2024-03-04T17:50:54.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.497+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~C3X6F0TD8UJS.0 size=1048576 2024-03-04T17:50:54.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.501+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~C3X6F0TD8UJS.1 size=956147 2024-03-04T17:50:54.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.509+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=C3X6F0TD8UJS tag=HLNUY2VWWV4T type=write size=3053299 2024-03-04T17:50:54.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.513+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=0CJ1FDRODJDY type=del size=1506501 2024-03-04T17:50:54.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.517+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~6FFC401ZDPY9.0 size=1048576 2024-03-04T17:50:54.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.521+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~6FFC401ZDPY9.1 size=638459 2024-03-04T17:50:54.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.525+0000 7fda08af98c0 1 < completed multipart upload key=C1GK00FM2ZYA upload=6FFC401ZDPY9 tag=2R0CNQ42SLNY type=write size=2735611 2024-03-04T17:50:54.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.529+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=ZW7G575WRA15 type=write size=214324 2024-03-04T17:50:54.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.533+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~USKN6SFZBXYF.0 size=1048576 2024-03-04T17:50:54.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.537+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~USKN6SFZBXYF.1 size=995205 2024-03-04T17:50:54.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.545+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=USKN6SFZBXYF tag=OOYF9DNTCYBD type=write size=3092357 2024-03-04T17:50:54.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.549+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~W814NY7KLX8D.0 size=1048576 2024-03-04T17:50:54.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.553+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~W814NY7KLX8D.1 size=1048576 2024-03-04T17:50:54.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.557+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~W814NY7KLX8D.2 size=680277 2024-03-04T17:50:54.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.557+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=W814NY7KLX8D tag=IHRJ5RVTZ59Z type=write size=3826005 2024-03-04T17:50:54.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.565+0000 7fda08af98c0 1 < suggested remove operation key=V1LDB2HXZ7EF tag=IWBE25YQ3RNH type=del 2024-03-04T17:50:54.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.569+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~ALRWFG6XJRIN.0 size=1048576 2024-03-04T17:50:54.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.573+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~ALRWFG6XJRIN.1 size=190936 2024-03-04T17:50:54.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.573+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=ALRWFG6XJRIN tag=6ZJMS2W0Y71S type=write size=2288088 2024-03-04T17:50:54.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.577+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=7EBLI3FQ30C0 type=del 2024-03-04T17:50:54.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.585+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~Y2J6BCPBVXB6.0 size=1048576 2024-03-04T17:50:54.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.589+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~Y2J6BCPBVXB6.1 size=1048576 2024-03-04T17:50:54.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.589+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~Y2J6BCPBVXB6.2 size=820503 2024-03-04T17:50:54.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.589+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=Y2J6BCPBVXB6 tag=XW5JXM4FTK2K type=write size=3966231 2024-03-04T17:50:54.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.593+0000 7fda08af98c0 1 < canceled operation key=AO3QS6CM7I5D tag=OGLFCBBH4WI2 type=del size=2827160 2024-03-04T17:50:54.600 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.601+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~21T7TVKFJFQ3.0 size=1048576 2024-03-04T17:50:54.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.605+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~21T7TVKFJFQ3.1 size=224967 2024-03-04T17:50:54.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.605+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=21T7TVKFJFQ3 tag=IJGMV23GMIZI type=write size=2322119 2024-03-04T17:50:54.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.609+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~CQFTIB1FA0UH.0 size=1048576 2024-03-04T17:50:54.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.613+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~CQFTIB1FA0UH.1 size=1048576 2024-03-04T17:50:54.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.613+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~CQFTIB1FA0UH.2 size=308965 2024-03-04T17:50:54.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.621+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=CQFTIB1FA0UH tag=YQW038IQY8EJ type=write size=3454693 2024-03-04T17:50:54.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.629+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~E4R1ABJPESW8.0 size=1048576 2024-03-04T17:50:54.628 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.629+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~E4R1ABJPESW8.1 size=687142 2024-03-04T17:50:54.629 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.629+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=E4R1ABJPESW8 tag=1TZWVSQ3QVCZ type=write size=2784294 2024-03-04T17:50:54.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.637+0000 7fda08af98c0 1 < suggested update operation key=AO3QS6CM7I5D tag=156AC5AIOMOX type=del size=1392265 2024-03-04T17:50:54.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.645+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~R02XSHF2L72V.0 size=214648 2024-03-04T17:50:54.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.645+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=R02XSHF2L72V tag=ESDX3K4LLERC type=write size=1263224 2024-03-04T17:50:54.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.649+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=HT9OG0ARB7L5 type=del 2024-03-04T17:50:54.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.657+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~4VAEYY7YMIJA.0 size=1048576 2024-03-04T17:50:54.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.661+0000 7fda08af98c0 1 > prepared part key=_multipart_09KBCA2XURTF.2~4VAEYY7YMIJA.1 size=151087 2024-03-04T17:50:54.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.661+0000 7fda08af98c0 1 > prepared multipart upload key=09KBCA2XURTF upload=4VAEYY7YMIJA tag=IGGBLNY1UT0N type=write size=2248239 2024-03-04T17:50:54.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.665+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~55D2L02EU202.0 size=1048576 2024-03-04T17:50:54.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.665+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~55D2L02EU202.1 size=1048576 2024-03-04T17:50:54.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.669+0000 7fda08af98c0 1 < completed part key=_multipart_L0FVKWD7EQRW.2~55D2L02EU202.2 size=635743 2024-03-04T17:50:54.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.677+0000 7fda08af98c0 1 < completed multipart upload key=L0FVKWD7EQRW upload=55D2L02EU202 tag=P251SK06IHFZ type=write size=3781471 2024-03-04T17:50:54.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.681+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=TICF4971FV2U type=del size=3363578 2024-03-04T17:50:54.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.685+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~TVU59EYZVTY7.0 size=1048576 2024-03-04T17:50:54.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.685+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~TVU59EYZVTY7.1 size=1044268 2024-03-04T17:50:54.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.693+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=TVU59EYZVTY7 tag=PQCUDS0KH5AL type=write size=3141420 2024-03-04T17:50:54.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.701+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~BTP9IW2MDM8L.0 size=1048576 2024-03-04T17:50:54.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.701+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~BTP9IW2MDM8L.1 size=818766 2024-03-04T17:50:54.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.701+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=BTP9IW2MDM8L tag=MVB3W8QQNDSL type=write size=2915918 2024-03-04T17:50:54.705 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.705+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~9U9R29E8OZM0.0 size=1048576 2024-03-04T17:50:54.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.709+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~9U9R29E8OZM0.1 size=56893 2024-03-04T17:50:54.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.717+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=9U9R29E8OZM0 tag=N7JLRMNLH8JZ type=write size=2154045 2024-03-04T17:50:54.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.721+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=1O1SKQKCOJPS type=del size=3223608 2024-03-04T17:50:54.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.721+0000 7fda08af98c0 1 < canceled part key=_multipart_IJSYAVR5SWPS.2~20THJ1ANX7RQ.0 size=1048576 2024-03-04T17:50:54.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.725+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~20THJ1ANX7RQ.1 size=1048576 2024-03-04T17:50:54.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.725+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~20THJ1ANX7RQ.2 size=410970 2024-03-04T17:50:54.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.733+0000 7fda08af98c0 1 < canceled multipart upload key=IJSYAVR5SWPS upload=20THJ1ANX7RQ tag=LIP9G47C22QN type=write size=3556698 2024-03-04T17:50:54.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.737+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=POA7U19P2I62 type=del size=4102777 2024-03-04T17:50:54.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.741+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~QTG25IO0OR7W.0 size=993333 2024-03-04T17:50:54.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.749+0000 7fda08af98c0 1 < completed multipart upload key=ITIVF3UAT9NK upload=QTG25IO0OR7W tag=M6XWEHWB4EKK type=write size=2041909 2024-03-04T17:50:54.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.753+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=WTBYIAOAFHMU type=del size=1188014 2024-03-04T17:50:54.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.761+0000 7fda08af98c0 1 < suggested update operation key=TOFWDFH0HL3Q tag=R06TNDNSWQBE type=del size=3291488 2024-03-04T17:50:54.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.765+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=C58WBPLLU4RZ type=write size=896726 2024-03-04T17:50:54.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.777+0000 7fda08af98c0 1 < suggested remove operation key=09KBCA2XURTF tag=NPN28S1VPAEN type=del 2024-03-04T17:50:54.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.781+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=TBTZGHY8ZA70 type=del size=123676 2024-03-04T17:50:54.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.785+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=0CJ1FDRODJDY type=del 2024-03-04T17:50:54.786 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.789+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=H8732KJY39ZU type=del size=2653478 2024-03-04T17:50:54.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.793+0000 7fda08af98c0 1 < completed write operation key=8YL7C9PI58TK tag=ZW7G575WRA15 type=write size=214324 2024-03-04T17:50:54.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.797+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~TSO833Q1N0FY.0 size=876362 2024-03-04T17:50:54.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.797+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=TSO833Q1N0FY tag=QBPZKLWMIMH8 type=write size=1924938 2024-03-04T17:50:54.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.801+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~W814NY7KLX8D.0 size=1048576 2024-03-04T17:50:54.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.805+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~W814NY7KLX8D.1 size=1048576 2024-03-04T17:50:54.806 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.809+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~W814NY7KLX8D.2 size=680277 2024-03-04T17:50:54.812 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.813+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=W814NY7KLX8D tag=IHRJ5RVTZ59Z type=write size=3826005 2024-03-04T17:50:54.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.821+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~FENISEFJFW7H.0 size=702049 2024-03-04T17:50:54.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.821+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=FENISEFJFW7H tag=JLS3S0BFEFNK type=write size=1750625 2024-03-04T17:50:54.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.825+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~ALRWFG6XJRIN.0 size=1048576 2024-03-04T17:50:54.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.825+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~ALRWFG6XJRIN.1 size=190936 2024-03-04T17:50:54.831 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.833+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=ALRWFG6XJRIN tag=6ZJMS2W0Y71S type=write size=2288088 2024-03-04T17:50:54.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.841+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~JU8DDMG1L0N1.0 size=348928 2024-03-04T17:50:54.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.841+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=JU8DDMG1L0N1 tag=4L2WKBQUSSDU type=write size=1397504 2024-03-04T17:50:54.841 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.845+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~Y2J6BCPBVXB6.0 size=1048576 2024-03-04T17:50:54.844 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.845+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~Y2J6BCPBVXB6.1 size=1048576 2024-03-04T17:50:54.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.849+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~Y2J6BCPBVXB6.2 size=820503 2024-03-04T17:50:54.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.857+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=Y2J6BCPBVXB6 tag=XW5JXM4FTK2K type=write size=3966231 2024-03-04T17:50:54.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.861+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~4679WYYG9BSG.0 size=1048576 2024-03-04T17:50:54.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.865+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~4679WYYG9BSG.1 size=1048576 2024-03-04T17:50:54.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.869+0000 7fda08af98c0 1 > prepared part key=_multipart_2PA8JDZUOLO5.2~4679WYYG9BSG.2 size=703921 2024-03-04T17:50:54.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.869+0000 7fda08af98c0 1 > prepared multipart upload key=2PA8JDZUOLO5 upload=4679WYYG9BSG tag=LCHMW1V8W4KB type=write size=3849649 2024-03-04T17:50:54.869 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.873+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~21T7TVKFJFQ3.0 size=1048576 2024-03-04T17:50:54.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.873+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~21T7TVKFJFQ3.1 size=224967 2024-03-04T17:50:54.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.881+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=21T7TVKFJFQ3 tag=IJGMV23GMIZI type=write size=2322119 2024-03-04T17:50:54.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.885+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=2G64Q4RE16BQ type=write size=118471 2024-03-04T17:50:54.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.889+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~E4R1ABJPESW8.0 size=1048576 2024-03-04T17:50:54.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.893+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~E4R1ABJPESW8.1 size=687142 2024-03-04T17:50:54.896 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.897+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=E4R1ABJPESW8 tag=1TZWVSQ3QVCZ type=write size=2784294 2024-03-04T17:50:54.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.901+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=TG1W5JD4I40N type=del size=4071434 2024-03-04T17:50:54.905 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.909+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~R02XSHF2L72V.0 size=214648 2024-03-04T17:50:54.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.913+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=R02XSHF2L72V tag=ESDX3K4LLERC type=write size=1263224 2024-03-04T17:50:54.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.917+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=JOWSBGAP39F7 type=del size=1623228 2024-03-04T17:50:54.920 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.921+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~4VAEYY7YMIJA.0 size=1048576 2024-03-04T17:50:54.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.925+0000 7fda08af98c0 1 < completed part key=_multipart_09KBCA2XURTF.2~4VAEYY7YMIJA.1 size=151087 2024-03-04T17:50:54.929 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.933+0000 7fda08af98c0 1 < completed multipart upload key=09KBCA2XURTF upload=4VAEYY7YMIJA tag=IGGBLNY1UT0N type=write size=2248239 2024-03-04T17:50:54.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.937+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=C9ZL3P1LY80L type=del size=2830023 2024-03-04T17:50:54.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.941+0000 7fda08af98c0 1 < canceled operation key=Z3J7W9V8HQKH tag=TICF4971FV2U type=del size=3363578 2024-03-04T17:50:54.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.945+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=YD3UBLGKVWBA type=write size=805835 2024-03-04T17:50:54.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.949+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~BTP9IW2MDM8L.0 size=1048576 2024-03-04T17:50:54.950 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.953+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~BTP9IW2MDM8L.1 size=818766 2024-03-04T17:50:54.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.957+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=BTP9IW2MDM8L tag=MVB3W8QQNDSL type=write size=2915918 2024-03-04T17:50:54.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.965+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~1FLOOY0FT8M5.0 size=1048576 2024-03-04T17:50:54.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.969+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~1FLOOY0FT8M5.1 size=1029376 2024-03-04T17:50:54.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.969+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=1FLOOY0FT8M5 tag=OOLDZ0KQYPNV type=write size=3126528 2024-03-04T17:50:54.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.973+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=1O1SKQKCOJPS type=del 2024-03-04T17:50:54.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.977+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=1RL6KG4TPRC7 type=del size=475716 2024-03-04T17:50:54.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.981+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=POA7U19P2I62 type=del 2024-03-04T17:50:54.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.985+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~SH67A1ABHLX9.0 size=1048576 2024-03-04T17:50:54.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.989+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~SH67A1ABHLX9.1 size=1048576 2024-03-04T17:50:54.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.993+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~SH67A1ABHLX9.2 size=584713 2024-03-04T17:50:54.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.993+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=SH67A1ABHLX9 tag=1Y07PCJHATLV type=write size=3730441 2024-03-04T17:50:54.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:54.997+0000 7fda08af98c0 1 < canceled operation key=8YL7C9PI58TK tag=WTBYIAOAFHMU type=del size=1188014 2024-03-04T17:50:54.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.001+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=YEATZX132L08 type=del size=3114648 2024-03-04T17:50:55.002 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.005+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=C58WBPLLU4RZ type=write size=896726 2024-03-04T17:50:55.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.009+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=R3TPPNPE4IZH type=del size=1231114 2024-03-04T17:50:55.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.013+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=TBTZGHY8ZA70 type=del 2024-03-04T17:50:55.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.017+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=7GS1YOAUWXED type=del size=148720 2024-03-04T17:50:55.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.021+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=H8732KJY39ZU type=del 2024-03-04T17:50:55.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.025+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=FSUY4JEKUWG3 type=write size=484665 2024-03-04T17:50:55.027 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.029+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~TSO833Q1N0FY.0 size=876362 2024-03-04T17:50:55.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.033+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=TSO833Q1N0FY tag=QBPZKLWMIMH8 type=write size=1924938 2024-03-04T17:50:55.040 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.041+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~CLL3LP9C3EPL.0 size=1048576 2024-03-04T17:50:55.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.045+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~CLL3LP9C3EPL.1 size=1048576 2024-03-04T17:50:55.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.049+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~CLL3LP9C3EPL.2 size=583016 2024-03-04T17:50:55.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.049+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=CLL3LP9C3EPL tag=0QFL7KJK23JR type=write size=3728744 2024-03-04T17:50:55.050 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.053+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~FENISEFJFW7H.0 size=702049 2024-03-04T17:50:55.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.057+0000 7fda08af98c0 1 < completed multipart upload key=XG5I3SDRE5JM upload=FENISEFJFW7H tag=JLS3S0BFEFNK type=write size=1750625 2024-03-04T17:50:55.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.061+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=694QR0XQ1VHL type=del size=3164906 2024-03-04T17:50:55.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.065+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~JU8DDMG1L0N1.0 size=348928 2024-03-04T17:50:55.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.073+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=JU8DDMG1L0N1 tag=4L2WKBQUSSDU type=write size=1397504 2024-03-04T17:50:55.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.077+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=B8QG0UOHOJC6 type=write size=76007 2024-03-04T17:50:55.077 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.081+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~4679WYYG9BSG.0 size=1048576 2024-03-04T17:50:55.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.081+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~4679WYYG9BSG.1 size=1048576 2024-03-04T17:50:55.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.085+0000 7fda08af98c0 1 < completed part key=_multipart_2PA8JDZUOLO5.2~4679WYYG9BSG.2 size=703921 2024-03-04T17:50:55.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.093+0000 7fda08af98c0 1 < completed multipart upload key=2PA8JDZUOLO5 upload=4679WYYG9BSG tag=LCHMW1V8W4KB type=write size=3849649 2024-03-04T17:50:55.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.097+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=6U18LVX3TRLH type=del size=1204069 2024-03-04T17:50:55.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.101+0000 7fda08af98c0 1 < completed write operation key=JA4JK99I0942 tag=2G64Q4RE16BQ type=write size=118471 2024-03-04T17:50:55.103 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.105+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=AOQH6LC6CF5Q type=write size=538358 2024-03-04T17:50:55.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.109+0000 7fda08af98c0 1 < canceled operation key=0IGSMFORCSL9 tag=TG1W5JD4I40N type=del size=4071434 2024-03-04T17:50:55.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.113+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=17NNKCVG08ED type=del size=919991 2024-03-04T17:50:55.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.121+0000 7fda08af98c0 1 < suggested remove operation key=XG2XYBCMUNY5 tag=JOWSBGAP39F7 type=del 2024-03-04T17:50:55.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.137+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~ZVLQB51RDRZO.0 size=703816 2024-03-04T17:50:55.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.137+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=ZVLQB51RDRZO tag=MK0NN6HCHZ3D type=write size=1752392 2024-03-04T17:50:55.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.145+0000 7fda08af98c0 1 < suggested remove operation key=750NSVFBS03A tag=C9ZL3P1LY80L type=del 2024-03-04T17:50:55.148 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.149+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=225UCF6BNEIC type=del size=3320951 2024-03-04T17:50:55.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.153+0000 7fda08af98c0 1 < completed write operation key=2PA8JDZUOLO5 tag=YD3UBLGKVWBA type=write size=805835 2024-03-04T17:50:55.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.157+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=XNJWX2Y0UNCU type=del size=3190594 2024-03-04T17:50:55.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.157+0000 7fda08af98c0 1 < canceled part key=_multipart_1OZKPHMPV3OQ.2~1FLOOY0FT8M5.0 size=1048576 2024-03-04T17:50:55.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.161+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~1FLOOY0FT8M5.1 size=1029376 2024-03-04T17:50:55.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.169+0000 7fda08af98c0 1 < canceled multipart upload key=1OZKPHMPV3OQ upload=1FLOOY0FT8M5 tag=OOLDZ0KQYPNV type=write size=3126528 2024-03-04T17:50:55.170 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.173+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=S0FTY3DYB1D8 type=del size=2229502 2024-03-04T17:50:55.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.177+0000 7fda08af98c0 -1 < failed to complete operation key=750NSVFBS03A tag=1RL6KG4TPRC7 type=del size=475716: (22) Invalid argument 2024-03-04T17:50:55.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.181+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=0A7OFDZOOH8A type=del size=3109999 2024-03-04T17:50:55.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.185+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~SH67A1ABHLX9.0 size=1048576 2024-03-04T17:50:55.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.185+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~SH67A1ABHLX9.1 size=1048576 2024-03-04T17:50:55.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.189+0000 7fda08af98c0 1 < completed part key=_multipart_Z3J7W9V8HQKH.2~SH67A1ABHLX9.2 size=584713 2024-03-04T17:50:55.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.197+0000 7fda08af98c0 1 < completed multipart upload key=Z3J7W9V8HQKH upload=SH67A1ABHLX9 tag=1Y07PCJHATLV type=write size=3730441 2024-03-04T17:50:55.199 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.201+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=5H3HQ5E8Q89B type=del size=1793332 2024-03-04T17:50:55.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.205+0000 7fda08af98c0 1 < canceled operation key=VJVORUG95FLC tag=YEATZX132L08 type=del size=3114648 2024-03-04T17:50:55.206 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.209+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=9Z0O1T1ST2BQ type=write size=769581 2024-03-04T17:50:55.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.213+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=R3TPPNPE4IZH type=del 2024-03-04T17:50:55.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.217+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=6L327W2ZQ0D6 type=del size=3190873 2024-03-04T17:50:55.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.225+0000 7fda08af98c0 1 < suggested update operation key=V1LDB2HXZ7EF tag=7GS1YOAUWXED type=del size=148720 2024-03-04T17:50:55.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.233+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~2C067JINVLWT.0 size=1043698 2024-03-04T17:50:55.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.233+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=2C067JINVLWT tag=86LY58HBDAAJ type=write size=2092274 2024-03-04T17:50:55.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.233+0000 7fda08af98c0 1 < completed write operation key=M5BMX7B7RZYL tag=FSUY4JEKUWG3 type=write size=484665 2024-03-04T17:50:55.237 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.237+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=K4ASUD9P6C7B type=del size=1688741 2024-03-04T17:50:55.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.241+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~CLL3LP9C3EPL.0 size=1048576 2024-03-04T17:50:55.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.241+0000 7fda08af98c0 1 < canceled part key=_multipart_G0BHMSCO0XLF.2~CLL3LP9C3EPL.1 size=1048576 2024-03-04T17:50:55.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.245+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~CLL3LP9C3EPL.2 size=583016 2024-03-04T17:50:55.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.253+0000 7fda08af98c0 1 < canceled multipart upload key=G0BHMSCO0XLF upload=CLL3LP9C3EPL tag=0QFL7KJK23JR type=write size=3728744 2024-03-04T17:50:55.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.257+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~PH7JRZTIUFYS.0 size=561758 2024-03-04T17:50:55.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.257+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=PH7JRZTIUFYS tag=OFVY30BY1RRY type=write size=1610334 2024-03-04T17:50:55.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.265+0000 7fda08af98c0 1 < suggested update operation key=Z3J7W9V8HQKH tag=694QR0XQ1VHL type=del size=3164906 2024-03-04T17:50:55.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.273+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=C8GAOPZBBJDC type=del size=2283495 2024-03-04T17:50:55.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.277+0000 7fda08af98c0 1 < completed write operation key=2PA8JDZUOLO5 tag=B8QG0UOHOJC6 type=write size=76007 2024-03-04T17:50:55.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.281+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~O5PVI5O36Z8P.0 size=1044771 2024-03-04T17:50:55.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.281+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=O5PVI5O36Z8P tag=F2ZVX09973CN type=write size=2093347 2024-03-04T17:50:55.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.285+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=6U18LVX3TRLH type=del 2024-03-04T17:50:55.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.293+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~PLRD6PUNXXWR.0 size=1048576 2024-03-04T17:50:55.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.297+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~PLRD6PUNXXWR.1 size=266447 2024-03-04T17:50:55.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.297+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=PLRD6PUNXXWR tag=II2YIKFJFE5X type=write size=2363599 2024-03-04T17:50:55.297 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.301+0000 7fda08af98c0 1 < completed write operation key=XG5I3SDRE5JM tag=AOQH6LC6CF5Q type=write size=538358 2024-03-04T17:50:55.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.305+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=6ZQW3BE0625K type=write size=68949 2024-03-04T17:50:55.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.309+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=17NNKCVG08ED type=del 2024-03-04T17:50:55.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.317+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~9RDFZ49M2SCT.0 size=1048576 2024-03-04T17:50:55.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.317+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~9RDFZ49M2SCT.1 size=1048576 2024-03-04T17:50:55.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.321+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~9RDFZ49M2SCT.2 size=514236 2024-03-04T17:50:55.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.321+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=9RDFZ49M2SCT tag=J67SQXHMBCK9 type=write size=3659964 2024-03-04T17:50:55.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.325+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~ZVLQB51RDRZO.0 size=703816 2024-03-04T17:50:55.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.333+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=ZVLQB51RDRZO tag=MK0NN6HCHZ3D type=write size=1752392 2024-03-04T17:50:55.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.337+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=TMHBZXEUM6N7 type=del size=1533245 2024-03-04T17:50:55.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.341+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=225UCF6BNEIC type=del 2024-03-04T17:50:55.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.345+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=JOSMK6M878K5 type=del size=933115 2024-03-04T17:50:55.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.345+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=XNJWX2Y0UNCU type=del 2024-03-04T17:50:55.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.349+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=LPI95SIEAW2J type=del size=1798213 2024-03-04T17:50:55.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.353+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=S0FTY3DYB1D8 type=del 2024-03-04T17:50:55.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.357+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=21XD46SRTDJA type=del size=874609 2024-03-04T17:50:55.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.361+0000 7fda08af98c0 1 < completed delete operation key=DJCCQ6S0E7UG tag=0A7OFDZOOH8A type=del 2024-03-04T17:50:55.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.365+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=EKY08OXUO3XS type=del size=2523932 2024-03-04T17:50:55.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.369+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=5H3HQ5E8Q89B type=del 2024-03-04T17:50:55.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.377+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~BJZD1PFZ6RUM.0 size=1048576 2024-03-04T17:50:55.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.381+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~BJZD1PFZ6RUM.1 size=1048576 2024-03-04T17:50:55.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.381+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~BJZD1PFZ6RUM.2 size=718342 2024-03-04T17:50:55.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.381+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=BJZD1PFZ6RUM tag=BGQJBTV17IAG type=write size=3864070 2024-03-04T17:50:55.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.385+0000 7fda08af98c0 -1 < failed to complete operation key=V1LDB2HXZ7EF tag=9Z0O1T1ST2BQ type=write size=769581: (22) Invalid argument 2024-03-04T17:50:55.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.389+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=MW0HSXGH8RJU type=del size=2077420 2024-03-04T17:50:55.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.397+0000 7fda08af98c0 1 < suggested update operation key=1OZKPHMPV3OQ tag=6L327W2ZQ0D6 type=del size=3190873 2024-03-04T17:50:55.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.401+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=ZA1JG5SZ6HW9 type=del size=1978970 2024-03-04T17:50:55.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.413+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~2C067JINVLWT.0 size=1043698 2024-03-04T17:50:55.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.417+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=2C067JINVLWT tag=86LY58HBDAAJ type=write size=2092274 2024-03-04T17:50:55.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.425+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~SJT6WYLAYANN.0 size=1048576 2024-03-04T17:50:55.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.429+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~SJT6WYLAYANN.1 size=96221 2024-03-04T17:50:55.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.429+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=SJT6WYLAYANN tag=EB7BW6GEF0C4 type=write size=2193373 2024-03-04T17:50:55.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.433+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=K4ASUD9P6C7B type=del 2024-03-04T17:50:55.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.437+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=HD5UNU55SF86 type=del size=464123 2024-03-04T17:50:55.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.441+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~PH7JRZTIUFYS.0 size=561758 2024-03-04T17:50:55.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.449+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=PH7JRZTIUFYS tag=OFVY30BY1RRY type=write size=1610334 2024-03-04T17:50:55.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.457+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~S507K5N05315.0 size=1048576 2024-03-04T17:50:55.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.457+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~S507K5N05315.1 size=441421 2024-03-04T17:50:55.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.457+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=S507K5N05315 tag=7NJ82VTYH7XO type=write size=2538573 2024-03-04T17:50:55.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.465+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=C8GAOPZBBJDC type=del 2024-03-04T17:50:55.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.469+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=GEKOSN5JSTKV type=del size=1622963 2024-03-04T17:50:55.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.473+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~O5PVI5O36Z8P.0 size=1044771 2024-03-04T17:50:55.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.477+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=O5PVI5O36Z8P tag=F2ZVX09973CN type=write size=2093347 2024-03-04T17:50:55.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.481+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=ZFZZA34HYNIK type=del size=2512975 2024-03-04T17:50:55.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.485+0000 7fda08af98c0 1 < canceled part key=_multipart_C1GK00FM2ZYA.2~PLRD6PUNXXWR.0 size=1048576 2024-03-04T17:50:55.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.485+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~PLRD6PUNXXWR.1 size=266447 2024-03-04T17:50:55.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.493+0000 7fda08af98c0 1 < canceled multipart upload key=C1GK00FM2ZYA upload=PLRD6PUNXXWR tag=II2YIKFJFE5X type=write size=2363599 2024-03-04T17:50:55.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.501+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~MLPY66S4R3RQ.0 size=1048576 2024-03-04T17:50:55.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.501+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~MLPY66S4R3RQ.1 size=801648 2024-03-04T17:50:55.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.501+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=MLPY66S4R3RQ tag=JQG1VGSERG4I type=write size=2898800 2024-03-04T17:50:55.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.509+0000 7fda08af98c0 1 < canceled operation key=HCKJZUMAK0Z0 tag=6ZQW3BE0625K type=write size=68949 2024-03-04T17:50:55.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.513+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=4NHTYSZTDKJQ type=del size=2509775 2024-03-04T17:50:55.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.517+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~9RDFZ49M2SCT.0 size=1048576 2024-03-04T17:50:55.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.521+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~9RDFZ49M2SCT.1 size=1048576 2024-03-04T17:50:55.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.525+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~9RDFZ49M2SCT.2 size=514236 2024-03-04T17:50:55.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.529+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=9RDFZ49M2SCT tag=J67SQXHMBCK9 type=write size=3659964 2024-03-04T17:50:55.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.533+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=KGX1V2MGUU7L type=del size=3150797 2024-03-04T17:50:55.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.537+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=TMHBZXEUM6N7 type=del 2024-03-04T17:50:55.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.545+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~N5I5RYMW6U67.0 size=702715 2024-03-04T17:50:55.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.545+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=N5I5RYMW6U67 tag=AMFZR2VAIB0U type=write size=1751291 2024-03-04T17:50:55.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.545+0000 7fda08af98c0 1 < completed delete operation key=HCKJZUMAK0Z0 tag=JOSMK6M878K5 type=del 2024-03-04T17:50:55.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.553+0000 7fda08af98c0 1 > prepared part key=_multipart_Z3J7W9V8HQKH.2~62RHL7UM9LE5.0 size=94913 2024-03-04T17:50:55.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.553+0000 7fda08af98c0 1 > prepared multipart upload key=Z3J7W9V8HQKH upload=62RHL7UM9LE5 tag=RH5AQ5GOQV7R type=write size=1143489 2024-03-04T17:50:55.560 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.561+0000 7fda08af98c0 1 < suggested update operation key=8YL7C9PI58TK tag=LPI95SIEAW2J type=del size=1798213 2024-03-04T17:50:55.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.569+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~7INB790IXK7L.0 size=1048576 2024-03-04T17:50:55.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.573+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~7INB790IXK7L.1 size=67258 2024-03-04T17:50:55.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.573+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=7INB790IXK7L tag=QFAORDIRWX6S type=write size=2164410 2024-03-04T17:50:55.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.577+0000 7fda08af98c0 1 < canceled operation key=Z2JQBN84QQIB tag=21XD46SRTDJA type=del size=874609 2024-03-04T17:50:55.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.581+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=39DKN6FAD9E3 type=write size=280452 2024-03-04T17:50:55.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.585+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=EKY08OXUO3XS type=del 2024-03-04T17:50:55.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.593+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~AKLOQJPKD3E0.0 size=133890 2024-03-04T17:50:55.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.593+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=AKLOQJPKD3E0 tag=REC5K4Z6Y4C4 type=write size=1182466 2024-03-04T17:50:55.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.593+0000 7fda08af98c0 1 < canceled part key=_multipart_750NSVFBS03A.2~BJZD1PFZ6RUM.0 size=1048576 2024-03-04T17:50:55.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.597+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~BJZD1PFZ6RUM.1 size=1048576 2024-03-04T17:50:55.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.601+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~BJZD1PFZ6RUM.2 size=718342 2024-03-04T17:50:55.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.605+0000 7fda08af98c0 1 < canceled multipart upload key=750NSVFBS03A upload=BJZD1PFZ6RUM tag=BGQJBTV17IAG type=write size=3864070 2024-03-04T17:50:55.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.613+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~ZSXYJ3Y78HL6.0 size=1048576 2024-03-04T17:50:55.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.617+0000 7fda08af98c0 1 > prepared part key=_multipart_8YL7C9PI58TK.2~ZSXYJ3Y78HL6.1 size=861964 2024-03-04T17:50:55.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.617+0000 7fda08af98c0 1 > prepared multipart upload key=8YL7C9PI58TK upload=ZSXYJ3Y78HL6 tag=1YCFQ9UR1OGX type=write size=2959116 2024-03-04T17:50:55.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.621+0000 7fda08af98c0 1 < completed delete operation key=09KBCA2XURTF tag=MW0HSXGH8RJU type=del 2024-03-04T17:50:55.622 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.625+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=PS5DV8BD1DKI type=del size=147830 2024-03-04T17:50:55.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.633+0000 7fda08af98c0 1 < suggested remove operation key=BYQFZSJ3WH07 tag=ZA1JG5SZ6HW9 type=del 2024-03-04T17:50:55.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.637+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=95KS8BJBUR4M type=write size=687730 2024-03-04T17:50:55.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.641+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~SJT6WYLAYANN.0 size=1048576 2024-03-04T17:50:55.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.645+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~SJT6WYLAYANN.1 size=96221 2024-03-04T17:50:55.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.649+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=SJT6WYLAYANN tag=EB7BW6GEF0C4 type=write size=2193373 2024-03-04T17:50:55.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.653+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=0C1H220P5MQR type=del size=3043783 2024-03-04T17:50:55.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.657+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=HD5UNU55SF86 type=del 2024-03-04T17:50:55.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.661+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=JTCHTBVHV1D1 type=del size=1202319 2024-03-04T17:50:55.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.665+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~S507K5N05315.0 size=1048576 2024-03-04T17:50:55.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.669+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~S507K5N05315.1 size=441421 2024-03-04T17:50:55.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.673+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=S507K5N05315 tag=7NJ82VTYH7XO type=write size=2538573 2024-03-04T17:50:55.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.681+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=H2VXWNCDIRZK type=del size=3420497 2024-03-04T17:50:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.685+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=GEKOSN5JSTKV type=del 2024-03-04T17:50:55.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.689+0000 7fda08af98c0 1 > prepared operation key=G0BHMSCO0XLF tag=T900YDSR385G type=del size=2431806 2024-03-04T17:50:55.689 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.693+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=ZFZZA34HYNIK type=del 2024-03-04T17:50:55.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.697+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~UKZ3RKGC2GWT.0 size=1048576 2024-03-04T17:50:55.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.701+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~UKZ3RKGC2GWT.1 size=1029282 2024-03-04T17:50:55.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.701+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=UKZ3RKGC2GWT tag=ABB1OYT9U07A type=write size=3126434 2024-03-04T17:50:55.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.705+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~MLPY66S4R3RQ.0 size=1048576 2024-03-04T17:50:55.705 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.705+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~MLPY66S4R3RQ.1 size=801648 2024-03-04T17:50:55.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.713+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=MLPY66S4R3RQ tag=JQG1VGSERG4I type=write size=2898800 2024-03-04T17:50:55.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.721+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~IUYTD67V9NCB.0 size=1048576 2024-03-04T17:50:55.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.721+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~IUYTD67V9NCB.1 size=626318 2024-03-04T17:50:55.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.721+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=IUYTD67V9NCB tag=3AWLSNJNN98W type=write size=2723470 2024-03-04T17:50:55.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.725+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=4NHTYSZTDKJQ type=del 2024-03-04T17:50:55.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.733+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~BR1CU6PPFSUF.0 size=1048576 2024-03-04T17:50:55.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.737+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~BR1CU6PPFSUF.1 size=1048576 2024-03-04T17:50:55.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.737+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~BR1CU6PPFSUF.2 size=113592 2024-03-04T17:50:55.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.737+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=BR1CU6PPFSUF tag=0DAP733MLMGN type=write size=3259320 2024-03-04T17:50:55.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.741+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=KGX1V2MGUU7L type=del 2024-03-04T17:50:55.744 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.745+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=AJLFX0AWWQ50 type=del size=1338680 2024-03-04T17:50:55.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.749+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~N5I5RYMW6U67.0 size=702715 2024-03-04T17:50:55.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.757+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=N5I5RYMW6U67 tag=AMFZR2VAIB0U type=write size=1751291 2024-03-04T17:50:55.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.761+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~P9B0L79E70CN.0 size=1048576 2024-03-04T17:50:55.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.765+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~P9B0L79E70CN.1 size=964420 2024-03-04T17:50:55.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.765+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=P9B0L79E70CN tag=KY8VORXFFGCA type=write size=3061572 2024-03-04T17:50:55.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.765+0000 7fda08af98c0 1 < canceled part key=_multipart_Z3J7W9V8HQKH.2~62RHL7UM9LE5.0 size=94913 2024-03-04T17:50:55.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.773+0000 7fda08af98c0 1 < canceled multipart upload key=Z3J7W9V8HQKH upload=62RHL7UM9LE5 tag=RH5AQ5GOQV7R type=write size=1143489 2024-03-04T17:50:55.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.773+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=KG1QC1RCAHLW type=del size=3163878 2024-03-04T17:50:55.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.777+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~7INB790IXK7L.0 size=1048576 2024-03-04T17:50:55.780 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.781+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~7INB790IXK7L.1 size=67258 2024-03-04T17:50:55.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.793+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=7INB790IXK7L tag=QFAORDIRWX6S type=write size=2164410 2024-03-04T17:50:55.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.793+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=OZ5LXD60BOV5 type=del size=720310 2024-03-04T17:50:55.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.797+0000 7fda08af98c0 1 < completed write operation key=V1LDB2HXZ7EF tag=39DKN6FAD9E3 type=write size=280452 2024-03-04T17:50:55.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.801+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=BR9KRAVLLMBK type=del size=4056838 2024-03-04T17:50:55.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.805+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~AKLOQJPKD3E0.0 size=133890 2024-03-04T17:50:55.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.813+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=AKLOQJPKD3E0 tag=REC5K4Z6Y4C4 type=write size=1182466 2024-03-04T17:50:55.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.817+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=7UAPPJQ7B10N type=write size=630969 2024-03-04T17:50:55.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.821+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~ZSXYJ3Y78HL6.0 size=1048576 2024-03-04T17:50:55.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.825+0000 7fda08af98c0 1 < completed part key=_multipart_8YL7C9PI58TK.2~ZSXYJ3Y78HL6.1 size=861964 2024-03-04T17:50:55.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.829+0000 7fda08af98c0 1 < completed multipart upload key=8YL7C9PI58TK upload=ZSXYJ3Y78HL6 tag=1YCFQ9UR1OGX type=write size=2959116 2024-03-04T17:50:55.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.833+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=BB4Y1TY1YDM4 type=del size=125704 2024-03-04T17:50:55.838 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.837+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=PS5DV8BD1DKI type=del 2024-03-04T17:50:55.840 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.841+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=50D16F5ICUWL type=del size=3780178 2024-03-04T17:50:55.844 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.845+0000 7fda08af98c0 1 < completed write operation key=VJVORUG95FLC tag=95KS8BJBUR4M type=write size=687730 2024-03-04T17:50:55.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.849+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=AIPMDOFA3SFY type=del size=3970052 2024-03-04T17:50:55.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.853+0000 7fda08af98c0 1 < completed delete operation key=TOFWDFH0HL3Q tag=0C1H220P5MQR type=del 2024-03-04T17:50:55.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.857+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=JFDDAZA218EK type=write size=142787 2024-03-04T17:50:55.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.861+0000 7fda08af98c0 1 < canceled operation key=G0BHMSCO0XLF tag=JTCHTBVHV1D1 type=del size=1202319 2024-03-04T17:50:55.869 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.869+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=UIHOXYBFVU6R type=del size=2180740 2024-03-04T17:50:55.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.877+0000 7fda08af98c0 1 < completed delete operation key=3UWSL7NJLACY tag=H2VXWNCDIRZK type=del 2024-03-04T17:50:55.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.881+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=XO8TZPWGG633 type=del size=2504293 2024-03-04T17:50:55.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.885+0000 7fda08af98c0 1 < completed delete operation key=G0BHMSCO0XLF tag=T900YDSR385G type=del 2024-03-04T17:50:55.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.889+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~UP4BFI52LHT8.0 size=1048576 2024-03-04T17:50:55.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.893+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~UP4BFI52LHT8.1 size=1048576 2024-03-04T17:50:55.894 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.897+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~UP4BFI52LHT8.2 size=84525 2024-03-04T17:50:55.894 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.897+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=UP4BFI52LHT8 tag=KJ4AQ8PNLDWM type=write size=3230253 2024-03-04T17:50:55.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.901+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~UKZ3RKGC2GWT.0 size=1048576 2024-03-04T17:50:55.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.905+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~UKZ3RKGC2GWT.1 size=1029282 2024-03-04T17:50:55.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.909+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=UKZ3RKGC2GWT tag=ABB1OYT9U07A type=write size=3126434 2024-03-04T17:50:55.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.913+0000 7fda08af98c0 1 > prepared operation key=V1LDB2HXZ7EF tag=HF7MFQBT49TY type=del size=65387 2024-03-04T17:50:55.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.917+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~IUYTD67V9NCB.0 size=1048576 2024-03-04T17:50:55.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.921+0000 7fda08af98c0 1 < completed part key=_multipart_C4B45GL40IMZ.2~IUYTD67V9NCB.1 size=626318 2024-03-04T17:50:55.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.925+0000 7fda08af98c0 1 < completed multipart upload key=C4B45GL40IMZ upload=IUYTD67V9NCB tag=3AWLSNJNN98W type=write size=2723470 2024-03-04T17:50:55.928 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.929+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=R6F0JY04Q5UC type=del size=2599400 2024-03-04T17:50:55.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.933+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~BR1CU6PPFSUF.0 size=1048576 2024-03-04T17:50:55.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.933+0000 7fda08af98c0 1 < canceled part key=_multipart_JA4JK99I0942.2~BR1CU6PPFSUF.1 size=1048576 2024-03-04T17:50:55.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.937+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~BR1CU6PPFSUF.2 size=113592 2024-03-04T17:50:55.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.945+0000 7fda08af98c0 1 < canceled multipart upload key=JA4JK99I0942 upload=BR1CU6PPFSUF tag=0DAP733MLMGN type=write size=3259320 2024-03-04T17:50:55.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.949+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=GFQ18NAJGDZT type=del size=1827120 2024-03-04T17:50:55.953 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.957+0000 7fda08af98c0 1 < suggested update operation key=C4B45GL40IMZ tag=AJLFX0AWWQ50 type=del size=1338680 2024-03-04T17:50:55.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.961+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=99HUDT5Q9F5E type=del size=2431577 2024-03-04T17:50:55.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.965+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~P9B0L79E70CN.0 size=1048576 2024-03-04T17:50:55.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.965+0000 7fda08af98c0 1 < canceled part key=_multipart_GDGTJZ21TG32.2~P9B0L79E70CN.1 size=964420 2024-03-04T17:50:55.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.969+0000 7fda08af98c0 1 < canceled multipart upload key=GDGTJZ21TG32 upload=P9B0L79E70CN tag=KY8VORXFFGCA type=write size=3061572 2024-03-04T17:50:55.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.973+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=0BCI07GQQJHS type=del size=2649851 2024-03-04T17:50:55.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.977+0000 7fda08af98c0 -1 < failed to complete operation key=C4B45GL40IMZ tag=KG1QC1RCAHLW type=del size=3163878: (22) Invalid argument 2024-03-04T17:50:55.979 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.981+0000 7fda08af98c0 1 > prepared operation key=HCA7QLGM2O4O tag=MUO32W1Y382K type=del size=2034618 2024-03-04T17:50:55.982 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.985+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=OZ5LXD60BOV5 type=del 2024-03-04T17:50:55.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.989+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=86YMZQG2NRRA type=del size=2294198 2024-03-04T17:50:55.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:55.997+0000 7fda08af98c0 1 < suggested update operation key=JA4JK99I0942 tag=BR9KRAVLLMBK type=del size=4056838 2024-03-04T17:50:56.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.001+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~MCAQ7C8S067R.0 size=542209 2024-03-04T17:50:56.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.001+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=MCAQ7C8S067R tag=SW080O7G7J8W type=write size=1590785 2024-03-04T17:50:56.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.005+0000 7fda08af98c0 1 < completed write operation key=GDGTJZ21TG32 tag=7UAPPJQ7B10N type=write size=630969 2024-03-04T17:50:56.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.009+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=BD8AAA8QED1F type=del size=4067256 2024-03-04T17:50:56.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.013+0000 7fda08af98c0 1 < canceled operation key=HCA7QLGM2O4O tag=BB4Y1TY1YDM4 type=del size=125704 2024-03-04T17:50:56.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.021+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~HBRA4JUVA5RW.0 size=1048576 2024-03-04T17:50:56.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.025+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~HBRA4JUVA5RW.1 size=1048576 2024-03-04T17:50:56.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.025+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~HBRA4JUVA5RW.2 size=76302 2024-03-04T17:50:56.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.025+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=HBRA4JUVA5RW tag=K1ARCG2IPVLN type=write size=3222030 2024-03-04T17:50:56.028 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.029+0000 7fda08af98c0 1 < completed delete operation key=XG5I3SDRE5JM tag=50D16F5ICUWL type=del 2024-03-04T17:50:56.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.033+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=EEMJ0AAM2NRW type=del size=3162328 2024-03-04T17:50:56.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.037+0000 7fda08af98c0 1 < canceled operation key=L0FVKWD7EQRW tag=AIPMDOFA3SFY type=del size=3970052 2024-03-04T17:50:56.042 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.045+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~ENQW71GQ5ZXV.0 size=1048576 2024-03-04T17:50:56.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.049+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~ENQW71GQ5ZXV.1 size=1048576 2024-03-04T17:50:56.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.049+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~ENQW71GQ5ZXV.2 size=1033609 2024-03-04T17:50:56.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.049+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=ENQW71GQ5ZXV tag=EU6RXREWG88G type=write size=4179337 2024-03-04T17:50:56.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.053+0000 7fda08af98c0 1 < completed write operation key=FMPZ2548LBBF tag=JFDDAZA218EK type=write size=142787 2024-03-04T17:50:56.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.061+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~2N5KC2SBU5ZI.0 size=938758 2024-03-04T17:50:56.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.061+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=2N5KC2SBU5ZI tag=6BI8P1EOU6ZW type=write size=1987334 2024-03-04T17:50:56.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.065+0000 7fda08af98c0 -1 < failed to complete operation key=JA4JK99I0942 tag=UIHOXYBFVU6R type=del size=2180740: (22) Invalid argument 2024-03-04T17:50:56.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.069+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=W5L719DH1YXI type=del size=3621009 2024-03-04T17:50:56.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.073+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=XO8TZPWGG633 type=del 2024-03-04T17:50:56.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.077+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=HVV1J7PJZS86 type=write size=235303 2024-03-04T17:50:56.076 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.077+0000 7fda08af98c0 1 < canceled part key=_multipart_FMPZ2548LBBF.2~UP4BFI52LHT8.0 size=1048576 2024-03-04T17:50:56.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.081+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~UP4BFI52LHT8.1 size=1048576 2024-03-04T17:50:56.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.085+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~UP4BFI52LHT8.2 size=84525 2024-03-04T17:50:56.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.093+0000 7fda08af98c0 1 < canceled multipart upload key=FMPZ2548LBBF upload=UP4BFI52LHT8 tag=KJ4AQ8PNLDWM type=write size=3230253 2024-03-04T17:50:56.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.097+0000 7fda08af98c0 1 > prepared operation key=ITIVF3UAT9NK tag=RD38SBQPFSOT type=del size=147287 2024-03-04T17:50:56.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.101+0000 7fda08af98c0 1 < canceled operation key=V1LDB2HXZ7EF tag=HF7MFQBT49TY type=del size=65387 2024-03-04T17:50:56.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.105+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=WGI9D3P42M7W type=write size=601780 2024-03-04T17:50:56.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.109+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=R6F0JY04Q5UC type=del 2024-03-04T17:50:56.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.113+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~SZ67G0WZ1JB2.0 size=111289 2024-03-04T17:50:56.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.113+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=SZ67G0WZ1JB2 tag=FTFZFAVMUMIP type=write size=1159865 2024-03-04T17:50:56.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.117+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=GFQ18NAJGDZT type=del 2024-03-04T17:50:56.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.121+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=5ZVV6T0BJBNZ type=del size=1643994 2024-03-04T17:50:56.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.125+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=99HUDT5Q9F5E type=del 2024-03-04T17:50:56.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.129+0000 7fda08af98c0 1 > prepared operation key=750NSVFBS03A tag=YI9LZQ76QKY2 type=del size=3356311 2024-03-04T17:50:56.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.133+0000 7fda08af98c0 1 < completed delete operation key=C1GK00FM2ZYA tag=0BCI07GQQJHS type=del 2024-03-04T17:50:56.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.141+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~8E0O85SY6GOC.0 size=1048576 2024-03-04T17:50:56.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.145+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~8E0O85SY6GOC.1 size=1048576 2024-03-04T17:50:56.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.149+0000 7fda08af98c0 1 > prepared part key=_multipart_FNWA2N9U14HO.2~8E0O85SY6GOC.2 size=677586 2024-03-04T17:50:56.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.149+0000 7fda08af98c0 1 > prepared multipart upload key=FNWA2N9U14HO upload=8E0O85SY6GOC tag=VY8M54TFD9GK type=write size=3823314 2024-03-04T17:50:56.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.149+0000 7fda08af98c0 1 < completed delete operation key=HCA7QLGM2O4O tag=MUO32W1Y382K type=del 2024-03-04T17:50:56.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.157+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~RZNSODKZKBC3.0 size=956613 2024-03-04T17:50:56.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.157+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=RZNSODKZKBC3 tag=MXYCIO3CJ9OH type=write size=2005189 2024-03-04T17:50:56.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.165+0000 7fda08af98c0 1 < suggested remove operation key=DJCCQ6S0E7UG tag=86YMZQG2NRRA type=del 2024-03-04T17:50:56.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.169+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=DQRLMXXCTVZK type=del size=1120134 2024-03-04T17:50:56.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.173+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~MCAQ7C8S067R.0 size=542209 2024-03-04T17:50:56.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.181+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=MCAQ7C8S067R tag=SW080O7G7J8W type=write size=1590785 2024-03-04T17:50:56.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.189+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~5PBTSDVTM497.0 size=1048576 2024-03-04T17:50:56.189 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.193+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~5PBTSDVTM497.1 size=1048576 2024-03-04T17:50:56.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.193+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~5PBTSDVTM497.2 size=291270 2024-03-04T17:50:56.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.193+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=5PBTSDVTM497 tag=RUEZMCZ6BQ57 type=write size=3436998 2024-03-04T17:50:56.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.197+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=BD8AAA8QED1F type=del 2024-03-04T17:50:56.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.201+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=6EVKM08KWEFJ type=del size=2762598 2024-03-04T17:50:56.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.205+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~HBRA4JUVA5RW.0 size=1048576 2024-03-04T17:50:56.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.209+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~HBRA4JUVA5RW.1 size=1048576 2024-03-04T17:50:56.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.213+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~HBRA4JUVA5RW.2 size=76302 2024-03-04T17:50:56.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.217+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=HBRA4JUVA5RW tag=K1ARCG2IPVLN type=write size=3222030 2024-03-04T17:50:56.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.221+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=D73IFZQ7E684 type=del size=3432704 2024-03-04T17:50:56.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.225+0000 7fda08af98c0 1 < completed delete operation key=8YL7C9PI58TK tag=EEMJ0AAM2NRW type=del 2024-03-04T17:50:56.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.237+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~Y6TI5TTK4JPB.0 size=1048576 2024-03-04T17:50:56.236 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.237+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~Y6TI5TTK4JPB.1 size=1048576 2024-03-04T17:50:56.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.241+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~Y6TI5TTK4JPB.2 size=1038364 2024-03-04T17:50:56.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.241+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=Y6TI5TTK4JPB tag=1HI2PRR1B8WJ type=write size=4184092 2024-03-04T17:50:56.243 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.245+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~ENQW71GQ5ZXV.0 size=1048576 2024-03-04T17:50:56.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.249+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~ENQW71GQ5ZXV.1 size=1048576 2024-03-04T17:50:56.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.249+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~ENQW71GQ5ZXV.2 size=1033609 2024-03-04T17:50:56.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.257+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=ENQW71GQ5ZXV tag=EU6RXREWG88G type=write size=4179337 2024-03-04T17:50:56.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.261+0000 7fda08af98c0 1 > prepared operation key=1OZKPHMPV3OQ tag=IXAAPCWLAK9F type=del size=4080350 2024-03-04T17:50:56.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.265+0000 7fda08af98c0 1 < completed part key=_multipart_FMPZ2548LBBF.2~2N5KC2SBU5ZI.0 size=938758 2024-03-04T17:50:56.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.273+0000 7fda08af98c0 1 < completed multipart upload key=FMPZ2548LBBF upload=2N5KC2SBU5ZI tag=6BI8P1EOU6ZW type=write size=1987334 2024-03-04T17:50:56.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.277+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=QT5KYUPS437E type=del size=2979952 2024-03-04T17:50:56.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.281+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=W5L719DH1YXI type=del 2024-03-04T17:50:56.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.285+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~5SPAFX3V1A96.0 size=1048576 2024-03-04T17:50:56.287 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.289+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~5SPAFX3V1A96.1 size=584165 2024-03-04T17:50:56.287 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.289+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=5SPAFX3V1A96 tag=R4KZ9MK46YZV type=write size=2681317 2024-03-04T17:50:56.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.293+0000 7fda08af98c0 1 < completed write operation key=8YL7C9PI58TK tag=HVV1J7PJZS86 type=write size=235303 2024-03-04T17:50:56.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.301+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~N7F1363I819Q.0 size=1048576 2024-03-04T17:50:56.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.305+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~N7F1363I819Q.1 size=58030 2024-03-04T17:50:56.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.305+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=N7F1363I819Q tag=XB30IN0O6J3W type=write size=2155182 2024-03-04T17:50:56.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.313+0000 7fda08af98c0 1 < suggested update operation key=ITIVF3UAT9NK tag=RD38SBQPFSOT type=del size=147287 2024-03-04T17:50:56.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.317+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~LUIDI24SH8Z7.0 size=1048576 2024-03-04T17:50:56.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.321+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~LUIDI24SH8Z7.1 size=1048576 2024-03-04T17:50:56.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.325+0000 7fda08af98c0 1 > prepared part key=_multipart_750NSVFBS03A.2~LUIDI24SH8Z7.2 size=271306 2024-03-04T17:50:56.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.325+0000 7fda08af98c0 1 > prepared multipart upload key=750NSVFBS03A upload=LUIDI24SH8Z7 tag=1HP4DEKFHIDR type=write size=3417034 2024-03-04T17:50:56.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.329+0000 7fda08af98c0 1 < completed write operation key=VJVORUG95FLC tag=WGI9D3P42M7W type=write size=601780 2024-03-04T17:50:56.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.337+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~237I6T3N7C12.0 size=1048576 2024-03-04T17:50:56.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.337+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~237I6T3N7C12.1 size=1048576 2024-03-04T17:50:56.340 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.341+0000 7fda08af98c0 1 > prepared part key=_multipart_RXVSQVVNXVF0.2~237I6T3N7C12.2 size=226103 2024-03-04T17:50:56.340 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.341+0000 7fda08af98c0 1 > prepared multipart upload key=RXVSQVVNXVF0 upload=237I6T3N7C12 tag=IMQF3SW9FZ92 type=write size=3371831 2024-03-04T17:50:56.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.345+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~SZ67G0WZ1JB2.0 size=111289 2024-03-04T17:50:56.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.353+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=SZ67G0WZ1JB2 tag=FTFZFAVMUMIP type=write size=1159865 2024-03-04T17:50:56.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.357+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=Z09HR4ZYWL7F type=write size=725404 2024-03-04T17:50:56.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.361+0000 7fda08af98c0 1 < canceled operation key=C4B45GL40IMZ tag=5ZVV6T0BJBNZ type=del size=1643994 2024-03-04T17:50:56.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.365+0000 7fda08af98c0 1 > prepared operation key=C1GK00FM2ZYA tag=JFWNN00AI9UR type=write size=106629 2024-03-04T17:50:56.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.369+0000 7fda08af98c0 1 < completed delete operation key=750NSVFBS03A tag=YI9LZQ76QKY2 type=del 2024-03-04T17:50:56.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.373+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=7BFN958JF0IT type=del size=1587863 2024-03-04T17:50:56.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.377+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~8E0O85SY6GOC.0 size=1048576 2024-03-04T17:50:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.377+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~8E0O85SY6GOC.1 size=1048576 2024-03-04T17:50:56.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.381+0000 7fda08af98c0 1 < completed part key=_multipart_FNWA2N9U14HO.2~8E0O85SY6GOC.2 size=677586 2024-03-04T17:50:56.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.389+0000 7fda08af98c0 1 < completed multipart upload key=FNWA2N9U14HO upload=8E0O85SY6GOC tag=VY8M54TFD9GK type=write size=3823314 2024-03-04T17:50:56.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.393+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=UQMMH9X1OCEV type=write size=825640 2024-03-04T17:50:56.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.397+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~RZNSODKZKBC3.0 size=956613 2024-03-04T17:50:56.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.401+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=RZNSODKZKBC3 tag=MXYCIO3CJ9OH type=write size=2005189 2024-03-04T17:50:56.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.405+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=0S0NQ30608BO type=del size=3110471 2024-03-04T17:50:56.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.409+0000 7fda08af98c0 1 < completed delete operation key=2PA8JDZUOLO5 tag=DQRLMXXCTVZK type=del 2024-03-04T17:50:56.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.413+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=FGXGJNX16MKM type=del size=1859040 2024-03-04T17:50:56.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.417+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~5PBTSDVTM497.0 size=1048576 2024-03-04T17:50:56.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.421+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~5PBTSDVTM497.1 size=1048576 2024-03-04T17:50:56.420 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.421+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~5PBTSDVTM497.2 size=291270 2024-03-04T17:50:56.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.429+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=5PBTSDVTM497 tag=RUEZMCZ6BQ57 type=write size=3436998 2024-03-04T17:50:56.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.437+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~ZJ6JY2RKV4BQ.0 size=1048576 2024-03-04T17:50:56.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.437+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~ZJ6JY2RKV4BQ.1 size=1048576 2024-03-04T17:50:56.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.441+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~ZJ6JY2RKV4BQ.2 size=661458 2024-03-04T17:50:56.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.441+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=ZJ6JY2RKV4BQ tag=K2575KAULQFJ type=write size=3807186 2024-03-04T17:50:56.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.445+0000 7fda08af98c0 1 < completed delete operation key=C4B45GL40IMZ tag=6EVKM08KWEFJ type=del 2024-03-04T17:50:56.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.453+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~VSLT217P5TEC.0 size=1048576 2024-03-04T17:50:56.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.457+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~VSLT217P5TEC.1 size=1048576 2024-03-04T17:50:56.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.457+0000 7fda08af98c0 1 > prepared part key=_multipart_DJCCQ6S0E7UG.2~VSLT217P5TEC.2 size=621207 2024-03-04T17:50:56.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.457+0000 7fda08af98c0 1 > prepared multipart upload key=DJCCQ6S0E7UG upload=VSLT217P5TEC tag=CIPG7KTK5PSW type=write size=3766935 2024-03-04T17:50:56.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.465+0000 7fda08af98c0 1 < canceled operation key=2PA8JDZUOLO5 tag=D73IFZQ7E684 type=del size=3432704 2024-03-04T17:50:56.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.469+0000 7fda08af98c0 1 > prepared part key=_multipart_XG2XYBCMUNY5.2~FN8813GYRVW3.0 size=42368 2024-03-04T17:50:56.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.469+0000 7fda08af98c0 1 > prepared multipart upload key=XG2XYBCMUNY5 upload=FN8813GYRVW3 tag=3XE48F2EB5EW type=write size=1090944 2024-03-04T17:50:56.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.473+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~Y6TI5TTK4JPB.0 size=1048576 2024-03-04T17:50:56.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.477+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~Y6TI5TTK4JPB.1 size=1048576 2024-03-04T17:50:56.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.481+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~Y6TI5TTK4JPB.2 size=1038364 2024-03-04T17:50:56.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.489+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=Y6TI5TTK4JPB tag=1HI2PRR1B8WJ type=write size=4184092 2024-03-04T17:50:56.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.497+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~L9Y19Y0KVCS6.0 size=1048576 2024-03-04T17:50:56.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.497+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~L9Y19Y0KVCS6.1 size=1048576 2024-03-04T17:50:56.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.501+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~L9Y19Y0KVCS6.2 size=170481 2024-03-04T17:50:56.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.501+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=L9Y19Y0KVCS6 tag=WM20JW3N3H11 type=write size=3316209 2024-03-04T17:50:56.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.505+0000 7fda08af98c0 1 < completed delete operation key=1OZKPHMPV3OQ tag=IXAAPCWLAK9F type=del 2024-03-04T17:50:56.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.509+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=3A8VGAIFXL5N type=del size=1424231 2024-03-04T17:50:56.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.513+0000 7fda08af98c0 1 < completed delete operation key=0IGSMFORCSL9 tag=QT5KYUPS437E type=del 2024-03-04T17:50:56.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.517+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=8UZDRI9S3VC7 type=del size=2699319 2024-03-04T17:50:56.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.521+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~5SPAFX3V1A96.0 size=1048576 2024-03-04T17:50:56.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.525+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~5SPAFX3V1A96.1 size=584165 2024-03-04T17:50:56.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.529+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=5SPAFX3V1A96 tag=R4KZ9MK46YZV type=write size=2681317 2024-03-04T17:50:56.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.533+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=4B970QDRMLX6 type=del size=1673483 2024-03-04T17:50:56.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.537+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~N7F1363I819Q.0 size=1048576 2024-03-04T17:50:56.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.541+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~N7F1363I819Q.1 size=58030 2024-03-04T17:50:56.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.545+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=N7F1363I819Q tag=XB30IN0O6J3W type=write size=2155182 2024-03-04T17:50:56.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.549+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=0XM9ZPT5QCH3 type=del size=1963919 2024-03-04T17:50:56.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.553+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~LUIDI24SH8Z7.0 size=1048576 2024-03-04T17:50:56.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.557+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~LUIDI24SH8Z7.1 size=1048576 2024-03-04T17:50:56.559 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.561+0000 7fda08af98c0 1 < completed part key=_multipart_750NSVFBS03A.2~LUIDI24SH8Z7.2 size=271306 2024-03-04T17:50:56.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.569+0000 7fda08af98c0 1 < completed multipart upload key=750NSVFBS03A upload=LUIDI24SH8Z7 tag=1HP4DEKFHIDR type=write size=3417034 2024-03-04T17:50:56.573 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.573+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~8Q9P5T3HR7IZ.0 size=1048576 2024-03-04T17:50:56.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.581+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~8Q9P5T3HR7IZ.1 size=1048576 2024-03-04T17:50:56.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.585+0000 7fda08af98c0 1 > prepared part key=_multipart_C1GK00FM2ZYA.2~8Q9P5T3HR7IZ.2 size=94391 2024-03-04T17:50:56.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.585+0000 7fda08af98c0 1 > prepared multipart upload key=C1GK00FM2ZYA upload=8Q9P5T3HR7IZ tag=CV2U6565VTXN type=write size=3240119 2024-03-04T17:50:56.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.589+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~237I6T3N7C12.0 size=1048576 2024-03-04T17:50:56.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.593+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~237I6T3N7C12.1 size=1048576 2024-03-04T17:50:56.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.597+0000 7fda08af98c0 1 < completed part key=_multipart_RXVSQVVNXVF0.2~237I6T3N7C12.2 size=226103 2024-03-04T17:50:56.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.601+0000 7fda08af98c0 1 < completed multipart upload key=RXVSQVVNXVF0 upload=237I6T3N7C12 tag=IMQF3SW9FZ92 type=write size=3371831 2024-03-04T17:50:56.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.609+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~AK4V02SJ68ID.0 size=606322 2024-03-04T17:50:56.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.609+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=AK4V02SJ68ID tag=MI00G2WIDBLX type=write size=1654898 2024-03-04T17:50:56.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.613+0000 7fda08af98c0 1 < completed write operation key=3UWSL7NJLACY tag=Z09HR4ZYWL7F type=write size=725404 2024-03-04T17:50:56.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.617+0000 7fda08af98c0 1 > prepared operation key=L0FVKWD7EQRW tag=CLAS2PCQHAKK type=del size=2507048 2024-03-04T17:50:56.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.621+0000 7fda08af98c0 1 < completed write operation key=C1GK00FM2ZYA tag=JFWNN00AI9UR type=write size=106629 2024-03-04T17:50:56.624 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.625+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=5J6V0G2ORVME type=del size=1409816 2024-03-04T17:50:56.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.633+0000 7fda08af98c0 1 < suggested remove operation key=2PA8JDZUOLO5 tag=7BFN958JF0IT type=del 2024-03-04T17:50:56.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.637+0000 7fda08af98c0 1 > prepared operation key=TOFWDFH0HL3Q tag=TWDZYRCE5Y7O type=write size=989328 2024-03-04T17:50:56.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.641+0000 7fda08af98c0 1 < completed write operation key=BYQFZSJ3WH07 tag=UQMMH9X1OCEV type=write size=825640 2024-03-04T17:50:56.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.649+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~JGHTHEIN009L.0 size=1048576 2024-03-04T17:50:56.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.653+0000 7fda08af98c0 1 > prepared part key=_multipart_XG5I3SDRE5JM.2~JGHTHEIN009L.1 size=176765 2024-03-04T17:50:56.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.653+0000 7fda08af98c0 1 > prepared multipart upload key=XG5I3SDRE5JM upload=JGHTHEIN009L tag=FLYSX78NX5QK type=write size=2273917 2024-03-04T17:50:56.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.657+0000 7fda08af98c0 1 < canceled operation key=C4B45GL40IMZ tag=0S0NQ30608BO type=del size=3110471 2024-03-04T17:50:56.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.665+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~ALT8UKU3ZKD1.0 size=1048576 2024-03-04T17:50:56.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.669+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~ALT8UKU3ZKD1.1 size=771708 2024-03-04T17:50:56.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.669+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=ALT8UKU3ZKD1 tag=KF15ZX3YPMFP type=write size=2868860 2024-03-04T17:50:56.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.673+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=FGXGJNX16MKM type=del 2024-03-04T17:50:56.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.677+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=FLB2UV5HF8LP type=del size=572679 2024-03-04T17:50:56.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.681+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~ZJ6JY2RKV4BQ.0 size=1048576 2024-03-04T17:50:56.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.685+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~ZJ6JY2RKV4BQ.1 size=1048576 2024-03-04T17:50:56.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.689+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~ZJ6JY2RKV4BQ.2 size=661458 2024-03-04T17:50:56.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.693+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=ZJ6JY2RKV4BQ tag=K2575KAULQFJ type=write size=3807186 2024-03-04T17:50:56.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.697+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=US47X7J2U5X3 type=write size=3039 2024-03-04T17:50:56.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.705+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~VSLT217P5TEC.0 size=1048576 2024-03-04T17:50:56.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.705+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~VSLT217P5TEC.1 size=1048576 2024-03-04T17:50:56.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.709+0000 7fda08af98c0 1 < completed part key=_multipart_DJCCQ6S0E7UG.2~VSLT217P5TEC.2 size=621207 2024-03-04T17:50:56.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.713+0000 7fda08af98c0 1 < completed multipart upload key=DJCCQ6S0E7UG upload=VSLT217P5TEC tag=CIPG7KTK5PSW type=write size=3766935 2024-03-04T17:50:56.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.717+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=RA9IZS1F5NL1 type=del size=1455953 2024-03-04T17:50:56.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.721+0000 7fda08af98c0 1 < completed part key=_multipart_XG2XYBCMUNY5.2~FN8813GYRVW3.0 size=42368 2024-03-04T17:50:56.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.729+0000 7fda08af98c0 1 < completed multipart upload key=XG2XYBCMUNY5 upload=FN8813GYRVW3 tag=3XE48F2EB5EW type=write size=1090944 2024-03-04T17:50:56.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.733+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=98VOAHUM1OZC type=del size=3355266 2024-03-04T17:50:56.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.737+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~L9Y19Y0KVCS6.0 size=1048576 2024-03-04T17:50:56.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.741+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~L9Y19Y0KVCS6.1 size=1048576 2024-03-04T17:50:56.742 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.745+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~L9Y19Y0KVCS6.2 size=170481 2024-03-04T17:50:56.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.749+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=L9Y19Y0KVCS6 tag=WM20JW3N3H11 type=write size=3316209 2024-03-04T17:50:56.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.753+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=06LUGYRH7TFP type=del size=2934923 2024-03-04T17:50:56.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.761+0000 7fda08af98c0 1 < completed delete operation key=IJSYAVR5SWPS tag=3A8VGAIFXL5N type=del 2024-03-04T17:50:56.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.765+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~QP0KU77BWZHB.0 size=1048576 2024-03-04T17:50:56.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.769+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~QP0KU77BWZHB.1 size=672408 2024-03-04T17:50:56.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.769+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=QP0KU77BWZHB tag=QI0HEK872Q18 type=write size=2769560 2024-03-04T17:50:56.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.781+0000 7fda08af98c0 1 < suggested remove operation key=8YL7C9PI58TK tag=8UZDRI9S3VC7 type=del 2024-03-04T17:50:56.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.785+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=6BOT4ICJVEXL type=del size=1194915 2024-03-04T17:50:56.786 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.789+0000 7fda08af98c0 1 < completed delete operation key=Z3J7W9V8HQKH tag=4B970QDRMLX6 type=del 2024-03-04T17:50:56.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.797+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~WL83J70E6YJT.0 size=1048576 2024-03-04T17:50:56.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.801+0000 7fda08af98c0 1 > prepared part key=_multipart_M5BMX7B7RZYL.2~WL83J70E6YJT.1 size=22507 2024-03-04T17:50:56.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.801+0000 7fda08af98c0 1 > prepared multipart upload key=M5BMX7B7RZYL upload=WL83J70E6YJT tag=TZP2YM19GR65 type=write size=2119659 2024-03-04T17:50:56.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.805+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=0XM9ZPT5QCH3 type=del size=1963919 2024-03-04T17:50:56.805 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.809+0000 7fda08af98c0 1 > prepared operation key=XG2XYBCMUNY5 tag=P4Z0VA4XW8WN type=del size=1048964 2024-03-04T17:50:56.806 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.809+0000 7fda08af98c0 1 < canceled part key=_multipart_C1GK00FM2ZYA.2~8Q9P5T3HR7IZ.0 size=1048576 2024-03-04T17:50:56.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.813+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~8Q9P5T3HR7IZ.1 size=1048576 2024-03-04T17:50:56.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.813+0000 7fda08af98c0 1 < completed part key=_multipart_C1GK00FM2ZYA.2~8Q9P5T3HR7IZ.2 size=94391 2024-03-04T17:50:56.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.821+0000 7fda08af98c0 1 < canceled multipart upload key=C1GK00FM2ZYA upload=8Q9P5T3HR7IZ tag=CV2U6565VTXN type=write size=3240119 2024-03-04T17:50:56.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.829+0000 7fda08af98c0 1 > prepared part key=_multipart_GDGTJZ21TG32.2~IKYLH6ZQXVU3.0 size=880172 2024-03-04T17:50:56.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.829+0000 7fda08af98c0 1 > prepared multipart upload key=GDGTJZ21TG32 upload=IKYLH6ZQXVU3 tag=18B1JP0I3WUA type=write size=1928748 2024-03-04T17:50:56.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.833+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~AK4V02SJ68ID.0 size=606322 2024-03-04T17:50:56.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.837+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=AK4V02SJ68ID tag=MI00G2WIDBLX type=write size=1654898 2024-03-04T17:50:56.842 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.845+0000 7fda08af98c0 1 > prepared part key=_multipart_V1LDB2HXZ7EF.2~H097WV5XW3NA.0 size=925251 2024-03-04T17:50:56.842 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.845+0000 7fda08af98c0 1 > prepared multipart upload key=V1LDB2HXZ7EF upload=H097WV5XW3NA tag=RIEDOP73LHTQ type=write size=1973827 2024-03-04T17:50:56.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.849+0000 7fda08af98c0 1 < completed delete operation key=L0FVKWD7EQRW tag=CLAS2PCQHAKK type=del 2024-03-04T17:50:56.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.853+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=QLQOWUGO9S9X type=del size=2260874 2024-03-04T17:50:56.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.857+0000 7fda08af98c0 1 < completed delete operation key=X5MEJ7G8X9HE tag=5J6V0G2ORVME type=del 2024-03-04T17:50:56.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.861+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~W8Q6M9RM17Q7.0 size=1048576 2024-03-04T17:50:56.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.865+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~W8Q6M9RM17Q7.1 size=1048576 2024-03-04T17:50:56.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.869+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~W8Q6M9RM17Q7.2 size=917607 2024-03-04T17:50:56.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.869+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=W8Q6M9RM17Q7 tag=POP8130CCBML type=write size=4063335 2024-03-04T17:50:56.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.873+0000 7fda08af98c0 1 < completed write operation key=TOFWDFH0HL3Q tag=TWDZYRCE5Y7O type=write size=989328 2024-03-04T17:50:56.875 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.877+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=L78U8B7R1UAR type=write size=40749 2024-03-04T17:50:56.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.881+0000 7fda08af98c0 1 < completed part key=_multipart_XG5I3SDRE5JM.2~JGHTHEIN009L.0 size=1048576 2024-03-04T17:50:56.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.881+0000 7fda08af98c0 1 < canceled part key=_multipart_XG5I3SDRE5JM.2~JGHTHEIN009L.1 size=176765 2024-03-04T17:50:56.885 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.885+0000 7fda08af98c0 1 < canceled multipart upload key=XG5I3SDRE5JM upload=JGHTHEIN009L tag=FLYSX78NX5QK type=write size=2273917 2024-03-04T17:50:56.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.889+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=1LMK6EGYAE8S type=del size=4008612 2024-03-04T17:50:56.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.893+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~ALT8UKU3ZKD1.0 size=1048576 2024-03-04T17:50:56.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.897+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~ALT8UKU3ZKD1.1 size=771708 2024-03-04T17:50:56.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.905+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=ALT8UKU3ZKD1 tag=KF15ZX3YPMFP type=write size=2868860 2024-03-04T17:50:56.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.909+0000 7fda08af98c0 1 > prepared part key=_multipart_AO3QS6CM7I5D.2~8XP9YBIZ97TY.0 size=878890 2024-03-04T17:50:56.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.909+0000 7fda08af98c0 1 > prepared multipart upload key=AO3QS6CM7I5D upload=8XP9YBIZ97TY tag=733EEGGFJKQ8 type=write size=1927466 2024-03-04T17:50:56.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.913+0000 7fda08af98c0 1 < completed delete operation key=RXVSQVVNXVF0 tag=FLB2UV5HF8LP type=del 2024-03-04T17:50:56.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.917+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=6DUQRN3DJQDP type=del size=1836449 2024-03-04T17:50:56.919 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.921+0000 7fda08af98c0 1 < completed write operation key=09KBCA2XURTF tag=US47X7J2U5X3 type=write size=3039 2024-03-04T17:50:56.923 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.925+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=K6CCBCLWEP32 type=del size=1697275 2024-03-04T17:50:56.927 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.929+0000 7fda08af98c0 1 < completed delete operation key=FNWA2N9U14HO tag=RA9IZS1F5NL1 type=del 2024-03-04T17:50:56.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.933+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=DSJWPTBPBLB0 type=del size=1573816 2024-03-04T17:50:56.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.941+0000 7fda08af98c0 1 < suggested update operation key=GDGTJZ21TG32 tag=98VOAHUM1OZC type=del size=3355266 2024-03-04T17:50:56.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.945+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=V89SNGRKTZ1A type=del size=842680 2024-03-04T17:50:56.955 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.957+0000 7fda08af98c0 1 < suggested update operation key=09KBCA2XURTF tag=06LUGYRH7TFP type=del size=2934923 2024-03-04T17:50:56.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.965+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~42QW4CQ4AG66.0 size=1048576 2024-03-04T17:50:56.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.965+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~42QW4CQ4AG66.1 size=1048576 2024-03-04T17:50:56.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.969+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~42QW4CQ4AG66.2 size=348083 2024-03-04T17:50:56.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.969+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=42QW4CQ4AG66 tag=LZRV52F7G3RJ type=write size=3493811 2024-03-04T17:50:56.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.973+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~QP0KU77BWZHB.0 size=1048576 2024-03-04T17:50:56.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.977+0000 7fda08af98c0 1 < completed part key=_multipart_AO3QS6CM7I5D.2~QP0KU77BWZHB.1 size=672408 2024-03-04T17:50:56.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.985+0000 7fda08af98c0 1 < completed multipart upload key=AO3QS6CM7I5D upload=QP0KU77BWZHB tag=QI0HEK872Q18 type=write size=2769560 2024-03-04T17:50:56.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.989+0000 7fda08af98c0 1 > prepared part key=_multipart_X5MEJ7G8X9HE.2~INS62OB18TNB.0 size=210790 2024-03-04T17:50:56.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.989+0000 7fda08af98c0 1 > prepared multipart upload key=X5MEJ7G8X9HE upload=INS62OB18TNB tag=B2G9OBFG0P6X type=write size=1259366 2024-03-04T17:50:56.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:56.993+0000 7fda08af98c0 -1 < failed to complete operation key=GDGTJZ21TG32 tag=6BOT4ICJVEXL type=del size=1194915: (22) Invalid argument 2024-03-04T17:50:56.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.001+0000 7fda08af98c0 1 > prepared part key=_multipart_HCA7QLGM2O4O.2~TX128HKEFK68.0 size=549172 2024-03-04T17:50:56.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.001+0000 7fda08af98c0 1 > prepared multipart upload key=HCA7QLGM2O4O upload=TX128HKEFK68 tag=7PN0B40YW0E8 type=write size=1597748 2024-03-04T17:50:57.002 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.005+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~WL83J70E6YJT.0 size=1048576 2024-03-04T17:50:57.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.005+0000 7fda08af98c0 1 < completed part key=_multipart_M5BMX7B7RZYL.2~WL83J70E6YJT.1 size=22507 2024-03-04T17:50:57.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.013+0000 7fda08af98c0 1 < completed multipart upload key=M5BMX7B7RZYL upload=WL83J70E6YJT tag=TZP2YM19GR65 type=write size=2119659 2024-03-04T17:50:57.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.021+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~U05HGNCK9UWK.0 size=917684 2024-03-04T17:50:57.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.021+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=U05HGNCK9UWK tag=XC2B2DB8YG20 type=write size=1966260 2024-03-04T17:50:57.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.025+0000 7fda08af98c0 1 < completed delete operation key=XG2XYBCMUNY5 tag=P4Z0VA4XW8WN type=del 2024-03-04T17:50:57.028 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.029+0000 7fda08af98c0 1 > prepared part key=_multipart_IJSYAVR5SWPS.2~N2YKMXO2EX0B.0 size=270326 2024-03-04T17:50:57.028 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.029+0000 7fda08af98c0 1 > prepared multipart upload key=IJSYAVR5SWPS upload=N2YKMXO2EX0B tag=XO2AYPM3H5WX type=write size=1318902 2024-03-04T17:50:57.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.033+0000 7fda08af98c0 1 < completed part key=_multipart_GDGTJZ21TG32.2~IKYLH6ZQXVU3.0 size=880172 2024-03-04T17:50:57.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.041+0000 7fda08af98c0 1 < completed multipart upload key=GDGTJZ21TG32 upload=IKYLH6ZQXVU3 tag=18B1JP0I3WUA type=write size=1928748 2024-03-04T17:50:57.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.045+0000 7fda08af98c0 1 > prepared operation key=BYQFZSJ3WH07 tag=WO40XXYSYWYL type=del size=2896512 2024-03-04T17:50:57.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.049+0000 7fda08af98c0 1 < completed part key=_multipart_V1LDB2HXZ7EF.2~H097WV5XW3NA.0 size=925251 2024-03-04T17:50:57.051 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.053+0000 7fda08af98c0 1 < completed multipart upload key=V1LDB2HXZ7EF upload=H097WV5XW3NA tag=RIEDOP73LHTQ type=write size=1973827 2024-03-04T17:50:57.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.057+0000 7fda08af98c0 1 > prepared operation key=HCKJZUMAK0Z0 tag=3OXPRCL118NL type=write size=940133 2024-03-04T17:50:57.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.061+0000 7fda08af98c0 1 < completed delete operation key=M5BMX7B7RZYL tag=QLQOWUGO9S9X type=del 2024-03-04T17:50:57.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.069+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~L4WIYXS1I6Y1.0 size=455887 2024-03-04T17:50:57.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.069+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=L4WIYXS1I6Y1 tag=90UPE2G24RJA type=write size=1504463 2024-03-04T17:50:57.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.073+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~W8Q6M9RM17Q7.0 size=1048576 2024-03-04T17:50:57.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.073+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~W8Q6M9RM17Q7.1 size=1048576 2024-03-04T17:50:57.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.073+0000 7fda08af98c0 1 < canceled part key=_multipart_X5MEJ7G8X9HE.2~W8Q6M9RM17Q7.2 size=917607 2024-03-04T17:50:57.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.081+0000 7fda08af98c0 1 < canceled multipart upload key=X5MEJ7G8X9HE upload=W8Q6M9RM17Q7 tag=POP8130CCBML type=write size=4063335 2024-03-04T17:50:57.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.085+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=KNGBK18L8D9N type=del size=627650 2024-03-04T17:50:57.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.093+0000 7fda08af98c0 1 < suggested update operation key=0IGSMFORCSL9 tag=L78U8B7R1UAR type=write size=40749 2024-03-04T17:50:57.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.097+0000 7fda08af98c0 1 > prepared operation key=0IGSMFORCSL9 tag=586OG9IX3K4L type=write size=867230 2024-03-04T17:50:57.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.105+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=1LMK6EGYAE8S type=del 2024-03-04T17:50:57.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.109+0000 7fda08af98c0 1 > prepared operation key=IJSYAVR5SWPS tag=KYAUNAI1XAE8 type=del size=4013247 2024-03-04T17:50:57.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.109+0000 7fda08af98c0 1 < canceled part key=_multipart_AO3QS6CM7I5D.2~8XP9YBIZ97TY.0 size=878890 2024-03-04T17:50:57.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.113+0000 7fda08af98c0 1 < canceled multipart upload key=AO3QS6CM7I5D upload=8XP9YBIZ97TY tag=733EEGGFJKQ8 type=write size=1927466 2024-03-04T17:50:57.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.117+0000 7fda08af98c0 1 > prepared operation key=M5BMX7B7RZYL tag=C9Q07H0E2K38 type=write size=958953 2024-03-04T17:50:57.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.125+0000 7fda08af98c0 1 < suggested update operation key=IJSYAVR5SWPS tag=6DUQRN3DJQDP type=del size=1836449 2024-03-04T17:50:57.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.133+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~2WF6IFHXO12L.0 size=1048576 2024-03-04T17:50:57.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.137+0000 7fda08af98c0 1 > prepared part key=_multipart_G0BHMSCO0XLF.2~2WF6IFHXO12L.1 size=780602 2024-03-04T17:50:57.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.137+0000 7fda08af98c0 1 > prepared multipart upload key=G0BHMSCO0XLF upload=2WF6IFHXO12L tag=SWQ980R51LW5 type=write size=2877754 2024-03-04T17:50:57.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.141+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=K6CCBCLWEP32 type=del 2024-03-04T17:50:57.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.149+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~284F30Q96VOL.0 size=1048576 2024-03-04T17:50:57.148 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.149+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~284F30Q96VOL.1 size=1048576 2024-03-04T17:50:57.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.153+0000 7fda08af98c0 1 > prepared part key=_multipart_ITIVF3UAT9NK.2~284F30Q96VOL.2 size=394207 2024-03-04T17:50:57.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.153+0000 7fda08af98c0 1 > prepared multipart upload key=ITIVF3UAT9NK upload=284F30Q96VOL tag=3NZVPKD04Y5R type=write size=3539935 2024-03-04T17:50:57.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.161+0000 7fda08af98c0 1 < suggested update operation key=HCKJZUMAK0Z0 tag=DSJWPTBPBLB0 type=del size=1573816 2024-03-04T17:50:57.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.165+0000 7fda08af98c0 1 > prepared operation key=8YL7C9PI58TK tag=1ANGAIHLOGFW type=del size=18102 2024-03-04T17:50:57.167 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.169+0000 7fda08af98c0 1 < completed delete operation key=Z2JQBN84QQIB tag=V89SNGRKTZ1A type=del 2024-03-04T17:50:57.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.173+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=9ID7LLFODKX5 type=write size=340555 2024-03-04T17:50:57.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.177+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~42QW4CQ4AG66.0 size=1048576 2024-03-04T17:50:57.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.181+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~42QW4CQ4AG66.1 size=1048576 2024-03-04T17:50:57.181 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.185+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~42QW4CQ4AG66.2 size=348083 2024-03-04T17:50:57.188 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.189+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=42QW4CQ4AG66 tag=LZRV52F7G3RJ type=write size=3493811 2024-03-04T17:50:57.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.193+0000 7fda08af98c0 1 > prepared operation key=VJVORUG95FLC tag=CYJ39VE31B4L type=write size=950913 2024-03-04T17:50:57.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.197+0000 7fda08af98c0 1 < completed part key=_multipart_X5MEJ7G8X9HE.2~INS62OB18TNB.0 size=210790 2024-03-04T17:50:57.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.205+0000 7fda08af98c0 1 < completed multipart upload key=X5MEJ7G8X9HE upload=INS62OB18TNB tag=B2G9OBFG0P6X type=write size=1259366 2024-03-04T17:50:57.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.213+0000 7fda08af98c0 1 > prepared part key=_multipart_VJVORUG95FLC.2~J2BLAGF23IFY.0 size=912460 2024-03-04T17:50:57.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.213+0000 7fda08af98c0 1 > prepared multipart upload key=VJVORUG95FLC upload=J2BLAGF23IFY tag=V6I0U0D2N7ZF type=write size=1961036 2024-03-04T17:50:57.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.217+0000 7fda08af98c0 1 < completed part key=_multipart_HCA7QLGM2O4O.2~TX128HKEFK68.0 size=549172 2024-03-04T17:50:57.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.221+0000 7fda08af98c0 1 < completed multipart upload key=HCA7QLGM2O4O upload=TX128HKEFK68 tag=7PN0B40YW0E8 type=write size=1597748 2024-03-04T17:50:57.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.225+0000 7fda08af98c0 1 > prepared operation key=FNWA2N9U14HO tag=F35FFZO8YP45 type=del size=3681547 2024-03-04T17:50:57.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.229+0000 7fda08af98c0 1 < completed part key=_multipart_1OZKPHMPV3OQ.2~U05HGNCK9UWK.0 size=917684 2024-03-04T17:50:57.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.237+0000 7fda08af98c0 1 < completed multipart upload key=1OZKPHMPV3OQ upload=U05HGNCK9UWK tag=XC2B2DB8YG20 type=write size=1966260 2024-03-04T17:50:57.241 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.241+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~K17R97EIYBPO.0 size=1048576 2024-03-04T17:50:57.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.245+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~K17R97EIYBPO.1 size=779731 2024-03-04T17:50:57.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.245+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=K17R97EIYBPO tag=TM1FSQKJ2581 type=write size=2876883 2024-03-04T17:50:57.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.249+0000 7fda08af98c0 1 < completed part key=_multipart_IJSYAVR5SWPS.2~N2YKMXO2EX0B.0 size=270326 2024-03-04T17:50:57.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.253+0000 7fda08af98c0 1 < completed multipart upload key=IJSYAVR5SWPS upload=N2YKMXO2EX0B tag=XO2AYPM3H5WX type=write size=1318902 2024-03-04T17:50:57.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.257+0000 7fda08af98c0 1 > prepared operation key=Z2JQBN84QQIB tag=195A1X6MMYU3 type=del size=2020892 2024-03-04T17:50:57.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.265+0000 7fda08af98c0 1 < suggested update operation key=BYQFZSJ3WH07 tag=WO40XXYSYWYL type=del size=2896512 2024-03-04T17:50:57.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.273+0000 7fda08af98c0 1 > prepared part key=_multipart_L0FVKWD7EQRW.2~505DEWLS6DZL.0 size=821693 2024-03-04T17:50:57.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.273+0000 7fda08af98c0 1 > prepared multipart upload key=L0FVKWD7EQRW upload=505DEWLS6DZL tag=5CB1TZ0QCRBT type=write size=1870269 2024-03-04T17:50:57.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.277+0000 7fda08af98c0 -1 < failed to complete operation key=HCKJZUMAK0Z0 tag=3OXPRCL118NL type=write size=940133: (22) Invalid argument 2024-03-04T17:50:57.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.285+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~RXAY2MHJXAZT.0 size=1048576 2024-03-04T17:50:57.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.285+0000 7fda08af98c0 1 > prepared part key=_multipart_3UWSL7NJLACY.2~RXAY2MHJXAZT.1 size=491145 2024-03-04T17:50:57.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.285+0000 7fda08af98c0 1 > prepared multipart upload key=3UWSL7NJLACY upload=RXAY2MHJXAZT tag=EP0WP4U8F9EH type=write size=2588297 2024-03-04T17:50:57.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.289+0000 7fda08af98c0 1 < completed part key=_multipart_BYQFZSJ3WH07.2~L4WIYXS1I6Y1.0 size=455887 2024-03-04T17:50:57.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.297+0000 7fda08af98c0 1 < completed multipart upload key=BYQFZSJ3WH07 upload=L4WIYXS1I6Y1 tag=90UPE2G24RJA type=write size=1504463 2024-03-04T17:50:57.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.305+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~3N8QLPB5QXXS.0 size=1048576 2024-03-04T17:50:57.308 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.309+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~3N8QLPB5QXXS.1 size=1048576 2024-03-04T17:50:57.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.313+0000 7fda08af98c0 1 > prepared part key=_multipart_JA4JK99I0942.2~3N8QLPB5QXXS.2 size=874306 2024-03-04T17:50:57.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.313+0000 7fda08af98c0 1 > prepared multipart upload key=JA4JK99I0942 upload=3N8QLPB5QXXS tag=APBAVB2AB0PS type=write size=4020034 2024-03-04T17:50:57.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.321+0000 7fda08af98c0 1 < suggested remove operation key=AO3QS6CM7I5D tag=KNGBK18L8D9N type=del 2024-03-04T17:50:57.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.325+0000 7fda08af98c0 1 > prepared operation key=AO3QS6CM7I5D tag=1HLMEE3O3A3D type=del size=3919758 2024-03-04T17:50:57.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.333+0000 7fda08af98c0 1 < suggested update operation key=0IGSMFORCSL9 tag=586OG9IX3K4L type=write size=867230 2024-03-04T17:50:57.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.337+0000 7fda08af98c0 1 > prepared operation key=3UWSL7NJLACY tag=1PAX84N21SSO type=write size=615292 2024-03-04T17:50:57.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.341+0000 7fda08af98c0 -1 < failed to complete operation key=IJSYAVR5SWPS tag=KYAUNAI1XAE8 type=del size=4013247: (22) Invalid argument 2024-03-04T17:50:57.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.345+0000 7fda08af98c0 1 > prepared operation key=FMPZ2548LBBF tag=SREO6XPHWD29 type=del size=3629959 2024-03-04T17:50:57.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.353+0000 7fda08af98c0 1 < suggested remove operation key=M5BMX7B7RZYL tag=C9Q07H0E2K38 type=write 2024-03-04T17:50:57.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.357+0000 7fda08af98c0 1 > prepared operation key=C4B45GL40IMZ tag=HFE0L3G488ZR type=del size=3790661 2024-03-04T17:50:57.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.357+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~2WF6IFHXO12L.0 size=1048576 2024-03-04T17:50:57.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.361+0000 7fda08af98c0 1 < completed part key=_multipart_G0BHMSCO0XLF.2~2WF6IFHXO12L.1 size=780602 2024-03-04T17:50:57.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.369+0000 7fda08af98c0 1 < completed multipart upload key=G0BHMSCO0XLF upload=2WF6IFHXO12L tag=SWQ980R51LW5 type=write size=2877754 2024-03-04T17:50:57.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.373+0000 7fda08af98c0 1 > prepared operation key=2PA8JDZUOLO5 tag=5PH283FZJVRM type=write size=108596 2024-03-04T17:50:57.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.373+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~284F30Q96VOL.0 size=1048576 2024-03-04T17:50:57.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.377+0000 7fda08af98c0 1 < completed part key=_multipart_ITIVF3UAT9NK.2~284F30Q96VOL.1 size=1048576 2024-03-04T17:50:57.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.377+0000 7fda08af98c0 1 < canceled part key=_multipart_ITIVF3UAT9NK.2~284F30Q96VOL.2 size=394207 2024-03-04T17:50:57.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.385+0000 7fda08af98c0 1 < canceled multipart upload key=ITIVF3UAT9NK upload=284F30Q96VOL tag=3NZVPKD04Y5R type=write size=3539935 2024-03-04T17:50:57.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.389+0000 7fda08af98c0 1 > prepared operation key=GDGTJZ21TG32 tag=32WY2FX7HUUA type=write size=576882 2024-03-04T17:50:57.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.393+0000 7fda08af98c0 1 < suggested update operation key=8YL7C9PI58TK tag=1ANGAIHLOGFW type=del size=18102 2024-03-04T17:50:57.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.401+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~J6ORNIU3ZJYH.0 size=1048576 2024-03-04T17:50:57.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.405+0000 7fda08af98c0 1 > prepared part key=_multipart_Z2JQBN84QQIB.2~J6ORNIU3ZJYH.1 size=688010 2024-03-04T17:50:57.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.405+0000 7fda08af98c0 1 > prepared multipart upload key=Z2JQBN84QQIB upload=J6ORNIU3ZJYH tag=HJA1WB1XIATE type=write size=2785162 2024-03-04T17:50:57.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.409+0000 7fda08af98c0 1 < completed write operation key=X5MEJ7G8X9HE tag=9ID7LLFODKX5 type=write size=340555 2024-03-04T17:50:57.410 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.413+0000 7fda08af98c0 1 > prepared operation key=DJCCQ6S0E7UG tag=AW0UJ02MLDRU type=del size=773195 2024-03-04T17:50:57.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.417+0000 7fda08af98c0 1 < suggested remove operation key=VJVORUG95FLC tag=CYJ39VE31B4L type=write 2024-03-04T17:50:57.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.421+0000 7fda08af98c0 1 > prepared operation key=XG5I3SDRE5JM tag=6RC6IOQXUWTF type=del size=961574 2024-03-04T17:50:57.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.425+0000 7fda08af98c0 1 < completed part key=_multipart_VJVORUG95FLC.2~J2BLAGF23IFY.0 size=912460 2024-03-04T17:50:57.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.433+0000 7fda08af98c0 1 < completed multipart upload key=VJVORUG95FLC upload=J2BLAGF23IFY tag=V6I0U0D2N7ZF type=write size=1961036 2024-03-04T17:50:57.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.437+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~UN1JYQIFWZGK.0 size=1048576 2024-03-04T17:50:57.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.441+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~UN1JYQIFWZGK.1 size=1048576 2024-03-04T17:50:57.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.445+0000 7fda08af98c0 1 > prepared part key=_multipart_BYQFZSJ3WH07.2~UN1JYQIFWZGK.2 size=113634 2024-03-04T17:50:57.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.445+0000 7fda08af98c0 1 > prepared multipart upload key=BYQFZSJ3WH07 upload=UN1JYQIFWZGK tag=S4E8X3C15DXF type=write size=3259362 2024-03-04T17:50:57.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.449+0000 7fda08af98c0 1 < canceled operation key=FNWA2N9U14HO tag=F35FFZO8YP45 type=del size=3681547 2024-03-04T17:50:57.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.453+0000 7fda08af98c0 1 > prepared operation key=Z3J7W9V8HQKH tag=8V9TUZJA6OBO type=del size=3772256 2024-03-04T17:50:57.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.453+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~K17R97EIYBPO.0 size=1048576 2024-03-04T17:50:57.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.457+0000 7fda08af98c0 1 < completed part key=_multipart_Z2JQBN84QQIB.2~K17R97EIYBPO.1 size=779731 2024-03-04T17:50:57.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.465+0000 7fda08af98c0 1 < completed multipart upload key=Z2JQBN84QQIB upload=K17R97EIYBPO tag=TM1FSQKJ2581 type=write size=2876883 2024-03-04T17:50:57.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.473+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~CLB2Y88C11GE.0 size=1048576 2024-03-04T17:50:57.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.473+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~CLB2Y88C11GE.1 size=1048576 2024-03-04T17:50:57.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.477+0000 7fda08af98c0 1 > prepared part key=_multipart_C4B45GL40IMZ.2~CLB2Y88C11GE.2 size=1017060 2024-03-04T17:50:57.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.477+0000 7fda08af98c0 1 > prepared multipart upload key=C4B45GL40IMZ upload=CLB2Y88C11GE tag=M5YAMQ3PP3Q7 type=write size=4162788 2024-03-04T17:50:57.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.485+0000 7fda08af98c0 1 < suggested update operation key=Z2JQBN84QQIB tag=195A1X6MMYU3 type=del size=2020892 2024-03-04T17:50:57.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.493+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~OXLW8X3DH1BS.0 size=1048576 2024-03-04T17:50:57.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.493+0000 7fda08af98c0 1 > prepared part key=_multipart_FMPZ2548LBBF.2~OXLW8X3DH1BS.1 size=627852 2024-03-04T17:50:57.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.493+0000 7fda08af98c0 1 > prepared multipart upload key=FMPZ2548LBBF upload=OXLW8X3DH1BS tag=LHBXJOWK84UN type=write size=2725004 2024-03-04T17:50:57.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.493+0000 7fda08af98c0 1 < canceled part key=_multipart_L0FVKWD7EQRW.2~505DEWLS6DZL.0 size=821693 2024-03-04T17:50:57.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.501+0000 7fda08af98c0 1 < canceled multipart upload key=L0FVKWD7EQRW upload=505DEWLS6DZL tag=5CB1TZ0QCRBT type=write size=1870269 2024-03-04T17:50:57.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.505+0000 7fda08af98c0 1 > prepared operation key=RXVSQVVNXVF0 tag=DKZOHZXP0A81 type=del size=2298580 2024-03-04T17:50:57.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.509+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~RXAY2MHJXAZT.0 size=1048576 2024-03-04T17:50:57.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.513+0000 7fda08af98c0 1 < completed part key=_multipart_3UWSL7NJLACY.2~RXAY2MHJXAZT.1 size=491145 2024-03-04T17:50:57.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.517+0000 7fda08af98c0 1 < completed multipart upload key=3UWSL7NJLACY upload=RXAY2MHJXAZT tag=EP0WP4U8F9EH type=write size=2588297 2024-03-04T17:50:57.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.521+0000 7fda08af98c0 1 > prepared operation key=X5MEJ7G8X9HE tag=61Z3VH2M67UK type=del size=2150700 2024-03-04T17:50:57.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.525+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~3N8QLPB5QXXS.0 size=1048576 2024-03-04T17:50:57.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.529+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~3N8QLPB5QXXS.1 size=1048576 2024-03-04T17:50:57.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.533+0000 7fda08af98c0 1 < completed part key=_multipart_JA4JK99I0942.2~3N8QLPB5QXXS.2 size=874306 2024-03-04T17:50:57.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.537+0000 7fda08af98c0 1 < completed multipart upload key=JA4JK99I0942 upload=3N8QLPB5QXXS tag=APBAVB2AB0PS type=write size=4020034 2024-03-04T17:50:57.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.541+0000 7fda08af98c0 1 > prepared operation key=09KBCA2XURTF tag=O079NBICN3NI type=del size=4119915 2024-03-04T17:50:57.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.545+0000 7fda08af98c0 1 < completed delete operation key=AO3QS6CM7I5D tag=1HLMEE3O3A3D type=del 2024-03-04T17:50:57.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.553+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~9TPYT73V50W1.0 size=1048576 2024-03-04T17:50:57.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.553+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~9TPYT73V50W1.1 size=592491 2024-03-04T17:50:57.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.553+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=9TPYT73V50W1 tag=ZXY4N265S95D type=write size=2689643 2024-03-04T17:50:57.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.557+0000 7fda08af98c0 1 < completed write operation key=3UWSL7NJLACY tag=1PAX84N21SSO type=write size=615292 2024-03-04T17:50:57.560 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.561+0000 7fda08af98c0 1 > prepared operation key=JA4JK99I0942 tag=RMGO190BHFCW type=del size=1656035 2024-03-04T17:50:57.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.565+0000 7fda08af98c0 1 < completed delete operation key=FMPZ2548LBBF tag=SREO6XPHWD29 type=del 2024-03-04T17:50:57.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.573+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~39WZRQ9LLT3R.0 size=1048576 2024-03-04T17:50:57.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.577+0000 7fda08af98c0 1 > prepared part key=_multipart_1OZKPHMPV3OQ.2~39WZRQ9LLT3R.1 size=85997 2024-03-04T17:50:57.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:50:57.577+0000 7fda08af98c0 1 > prepared multipart upload key=1OZKPHMPV3OQ upload=39WZRQ9LLT3R tag=WGVEG09VLZBI type=write size=2183149 2024-03-04T17:50:57.575 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] cls_rgw_stats.simulate (28368 ms) 2024-03-04T17:50:57.575 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 1 test from cls_rgw_stats (28368 ms total) 2024-03-04T17:50:57.575 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:50:57.575 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:50:58.275 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 1 test from 1 test suite ran. (30958 ms total) 2024-03-04T17:50:58.275 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 1 test. 2024-03-04T17:50:58.280 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:50:58.280 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:50:58.339 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2024-03-04T17:50:58.341 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2024-03-04T17:50:58.341 DEBUG:teuthology.orchestra.run.smithi047:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2024-03-04T17:50:58.405 INFO:tasks.workunit.client.0.smithi047.stdout:Running main() from gmock_main.cc 2024-03-04T17:50:58.405 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 31 tests from 1 test suite. 2024-03-04T17:50:58.405 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:51:00.277 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 31 tests from CmpOmap 2024-03-04T17:51:00.277 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2024-03-04T17:51:00.324 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (47 ms) 2024-03-04T17:51:00.324 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2024-03-04T17:51:00.328 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (4 ms) 2024-03-04T17:51:00.328 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2024-03-04T17:51:00.333 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (5 ms) 2024-03-04T17:51:00.333 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2024-03-04T17:51:00.340 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (7 ms) 2024-03-04T17:51:00.340 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_str 2024-03-04T17:51:00.350 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_str (10 ms) 2024-03-04T17:51:00.350 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2024-03-04T17:51:00.369 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_u64 (18 ms) 2024-03-04T17:51:00.369 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2024-03-04T17:51:00.377 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (8 ms) 2024-03-04T17:51:00.377 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2024-03-04T17:51:00.381 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (4 ms) 2024-03-04T17:51:00.381 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2024-03-04T17:51:00.592 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (211 ms) 2024-03-04T17:51:00.592 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2024-03-04T17:51:00.602 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (10 ms) 2024-03-04T17:51:00.602 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2024-03-04T17:51:00.604 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (2 ms) 2024-03-04T17:51:00.604 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2024-03-04T17:51:00.606 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (2 ms) 2024-03-04T17:51:00.606 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2024-03-04T17:51:00.622 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (16 ms) 2024-03-04T17:51:00.622 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2024-03-04T17:51:00.642 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (20 ms) 2024-03-04T17:51:00.642 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2024-03-04T17:51:00.659 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (17 ms) 2024-03-04T17:51:00.659 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2024-03-04T17:51:00.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (19 ms) 2024-03-04T17:51:00.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2024-03-04T17:51:00.702 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_str (24 ms) 2024-03-04T17:51:00.702 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2024-03-04T17:51:00.722 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (20 ms) 2024-03-04T17:51:00.722 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2024-03-04T17:51:00.728 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (6 ms) 2024-03-04T17:51:00.728 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2024-03-04T17:51:00.735 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (7 ms) 2024-03-04T17:51:00.735 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2024-03-04T17:51:00.737 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (3 ms) 2024-03-04T17:51:00.738 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2024-03-04T17:51:00.740 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (2 ms) 2024-03-04T17:51:00.740 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2024-03-04T17:51:00.758 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (18 ms) 2024-03-04T17:51:00.758 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2024-03-04T17:51:00.776 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (18 ms) 2024-03-04T17:51:00.776 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2024-03-04T17:51:00.797 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (21 ms) 2024-03-04T17:51:00.797 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2024-03-04T17:51:00.817 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (20 ms) 2024-03-04T17:51:00.817 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2024-03-04T17:51:00.823 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (6 ms) 2024-03-04T17:51:00.823 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2024-03-04T17:51:00.830 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (7 ms) 2024-03-04T17:51:00.830 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2024-03-04T17:51:00.833 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (3 ms) 2024-03-04T17:51:00.833 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2024-03-04T17:51:00.835 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (3 ms) 2024-03-04T17:51:00.835 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2024-03-04T17:51:00.857 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (21 ms) 2024-03-04T17:51:00.858 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 31 tests from CmpOmap (580 ms total) 2024-03-04T17:51:00.858 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:51:00.858 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:51:01.313 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 31 tests from 1 test suite ran. (2909 ms total) 2024-03-04T17:51:01.314 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 31 tests. 2024-03-04T17:51:01.321 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:51:01.321 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:51:01.379 INFO:tasks.workunit:Running workunits matching cls/test_cls_2pc_queue.sh on client.0... 2024-03-04T17:51:01.380 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2024-03-04T17:51:01.380 DEBUG:teuthology.orchestra.run.smithi047:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2024-03-04T17:51:01.447 INFO:tasks.workunit.client.0.smithi047.stdout:Running main() from gmock_main.cc 2024-03-04T17:51:01.447 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 24 tests from 1 test suite. 2024-03-04T17:51:01.447 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:51:01.447 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 24 tests from TestCls2PCQueue 2024-03-04T17:51:01.447 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2024-03-04T17:51:04.345 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2898 ms) 2024-03-04T17:51:04.345 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2024-03-04T17:51:07.368 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (3022 ms) 2024-03-04T17:51:07.368 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.Reserve 2024-03-04T17:51:10.431 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.Reserve (3064 ms) 2024-03-04T17:51:10.431 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2024-03-04T17:51:13.451 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3020 ms) 2024-03-04T17:51:13.451 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.Commit 2024-03-04T17:51:17.494 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.Commit (4043 ms) 2024-03-04T17:51:17.494 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.Stats 2024-03-04T17:51:21.548 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.Stats (4054 ms) 2024-03-04T17:51:21.548 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.UpgradeFromReef 2024-03-04T17:51:26.662 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.UpgradeFromReef (5114 ms) 2024-03-04T17:51:26.663 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.Abort 2024-03-04T17:51:29.719 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.Abort (3057 ms) 2024-03-04T17:51:29.719 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2024-03-04T17:51:32.745 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.ReserveError (3026 ms) 2024-03-04T17:51:32.745 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.CommitError 2024-03-04T17:51:35.787 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.CommitError (3043 ms) 2024-03-04T17:51:35.788 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.AbortError 2024-03-04T17:51:38.815 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.AbortError (3027 ms) 2024-03-04T17:51:38.815 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2024-03-04T17:51:41.858 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3044 ms) 2024-03-04T17:51:41.859 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2024-03-04T17:51:44.889 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3031 ms) 2024-03-04T17:51:44.890 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2024-03-04T17:51:47.921 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3031 ms) 2024-03-04T17:51:47.922 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2024-03-04T17:51:50.953 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3030 ms) 2024-03-04T17:51:50.953 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2024-03-04T17:51:53.993 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3042 ms) 2024-03-04T17:51:53.993 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2024-03-04T17:51:57.044 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3052 ms) 2024-03-04T17:51:57.044 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2024-03-04T17:52:00.088 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.Cleanup (3043 ms) 2024-03-04T17:52:00.088 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2024-03-04T17:52:15.153 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.MultiProducer (15060 ms) 2024-03-04T17:52:15.153 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2024-03-04T17:52:20.201 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (5053 ms) 2024-03-04T17:52:20.201 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2024-03-04T17:52:30.700 INFO:tasks.workunit.client.0.smithi047.stderr:Queue was never full - all reservations were successful.Please decrease the amount of consumer threads 2024-03-04T17:52:31.258 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (11058 ms) 2024-03-04T17:52:31.258 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2024-03-04T17:52:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (7049 ms) 2024-03-04T17:52:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2024-03-04T17:52:46.636 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (8323 ms) 2024-03-04T17:52:46.636 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2024-03-04T17:52:55.690 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (9059 ms) 2024-03-04T17:52:55.690 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 24 tests from TestCls2PCQueue (114243 ms total) 2024-03-04T17:52:55.690 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:52:55.690 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:55.690 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 24 tests from 1 test suite ran. (114243 ms total) 2024-03-04T17:52:55.690 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 24 tests. 2024-03-04T17:52:55.698 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:52:55.698 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:52:55.754 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2024-03-04T17:52:55.755 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2024-03-04T17:52:55.755 DEBUG:teuthology.orchestra.run.smithi047:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2024-03-04T17:52:55.832 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:55.832 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:55.832 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 6 tests from 1 test suite. 2024-03-04T17:52:55.832 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:57.727 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 6 tests from rgw_gc_log 2024-03-04T17:52:57.728 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2024-03-04T17:52:57.750 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] rgw_gc_log.init_existing_queue (23 ms) 2024-03-04T17:52:57.750 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2024-03-04T17:52:57.755 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] rgw_gc_log.init_existing_omap (5 ms) 2024-03-04T17:52:57.755 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2024-03-04T17:52:57.760 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (5 ms) 2024-03-04T17:52:57.760 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2024-03-04T17:52:57.762 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (2 ms) 2024-03-04T17:52:57.762 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2024-03-04T17:52:57.767 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] rgw_gc_log.defer1_after_init (4 ms) 2024-03-04T17:52:57.767 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2024-03-04T17:52:57.769 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] rgw_gc_log.defer2_before_init (3 ms) 2024-03-04T17:52:57.769 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 6 tests from rgw_gc_log (42 ms total) 2024-03-04T17:52:57.769 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:52:57.769 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:58.730 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 6 tests from 1 test suite ran. (2898 ms total) 2024-03-04T17:52:58.730 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 6 tests. 2024-03-04T17:52:58.735 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:52:58.735 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:52:58.790 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_obj.sh on client.0... 2024-03-04T17:52:58.791 INFO:tasks.workunit:Running workunit rgw/test_rgw_obj.sh... 2024-03-04T17:52:58.791 DEBUG:teuthology.orchestra.run.smithi047:workunit test rgw/test_rgw_obj.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_obj.sh 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:Running main() from gmock_main.cc 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 4 tests from TestRGWObj 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestRGWObj.underscore 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns= i= 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.863 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns= i= 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns= i= 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "obj1": { 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.864 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "obj2": { 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.865 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3": { 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3-2": { 2024-03-04T17:52:58.867 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.868 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns=ns i= 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns=ns i= 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns=ns i= 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout: "obj1": { 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.869 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.870 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "obj2": { 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.871 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3": { 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.872 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3-2": { 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.873 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns= i=v1 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns= i=v1 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns= i=v1 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "obj1": { 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.875 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "obj2": { 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.876 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3": { 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.877 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3-2": { 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.878 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.879 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns=ns i=v1 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns=ns i=v1 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: _obj[v1] 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: _obj ns=ns i=v1 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout: "obj1": { 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.880 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "obj2": { 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.881 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3": { 2024-03-04T17:52:58.882 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.883 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3-2": { 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_obj", 2024-03-04T17:52:58.884 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestRGWObj.underscore (1 ms) 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestRGWObj.no_underscore 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns= i= 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns= i= 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns= i= 2024-03-04T17:52:58.885 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "obj1": { 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.886 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "obj2": { 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.887 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3": { 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.888 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3-2": { 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.889 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns=ns i= 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.890 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns=ns i= 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns=ns i= 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "obj1": { 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.891 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "obj2": { 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.892 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3": { 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.893 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3-2": { 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.895 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns= i=v1 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns= i=v1 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns= i=v1 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout: "obj1": { 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.896 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "obj2": { 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.897 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3": { 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.898 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3-2": { 2024-03-04T17:52:58.899 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.900 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns=ns i=v1 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns=ns i=v1 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:parsed: obj[v1] 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:stripped: obj ns=ns i=v1 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout: "obj1": { 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.901 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "obj2": { 2024-03-04T17:52:58.902 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3": { 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.904 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "obj3-2": { 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.905 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "obj", 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "v1", 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestRGWObj.no_underscore (0 ms) 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestRGWObj.obj_to_raw 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.906 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.907 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.908 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker_myobj", 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.909 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.910 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.911 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker_myobj", 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.912 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.913 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.914 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.915 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns_myobj", 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.916 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.917 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.918 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.919 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns_myobj", 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.920 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.921 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst_myobj", 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.924 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.925 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.926 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst_myobj", 2024-03-04T17:52:58.927 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.929 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.930 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst_myobj", 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.931 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.932 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.933 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.934 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst_myobj", 2024-03-04T17:52:58.935 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.936 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.936 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.936 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.936 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.939 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.940 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.941 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.942 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker___myobj", 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj" 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.943 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.944 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.945 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker___myobj", 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj" 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.948 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.949 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.950 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns__myobj", 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.951 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.953 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns__myobj", 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.954 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.955 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.956 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst__myobj", 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj" 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.958 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.959 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.960 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.961 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst__myobj", 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj" 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.963 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.964 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.965 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst__myobj", 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.966 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.967 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.968 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.969 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst__myobj", 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.971 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.972 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.973 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker___myobj_", 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.974 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.976 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker___myobj_", 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.977 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.979 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.980 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns__myobj_", 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.981 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.984 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.985 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.985 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.985 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.985 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.985 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.985 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns__myobj_", 2024-03-04T17:52:58.985 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.986 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.987 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.988 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst__myobj_", 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.989 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.991 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.992 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst__myobj_", 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.993 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.994 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.995 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.996 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": "default-placement.data", 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst__myobj_", 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "test", 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.997 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": "", 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": "" 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "obj": { 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:58.999 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.000 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst__myobj_", 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.001 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestRGWObj.obj_to_raw (0 ms) 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] TestRGWObj.old_to_raw 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker_myobj", 2024-03-04T17:52:59.002 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.003 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker_myobj 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker_myobj", 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.004 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker_myobj 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns_myobj", 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.006 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker__ns_myobj 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.007 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns_myobj", 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.008 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker__ns_myobj 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst_myobj", 2024-03-04T17:52:59.009 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.010 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker__:inst_myobj 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst_myobj", 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.011 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.012 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker__:inst_myobj 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst_myobj", 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.013 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.014 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker__ns:inst_myobj 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst_myobj", 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.015 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "myobj", 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.016 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker__ns:inst_myobj 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker___myobj", 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj" 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.017 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker___myobj 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker___myobj", 2024-03-04T17:52:59.018 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj" 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.019 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker___myobj 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns__myobj", 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.020 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker__ns__myobj 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns__myobj", 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.022 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.023 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker__ns__myobj 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst__myobj", 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj" 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.024 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker__:inst__myobj 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.025 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst__myobj", 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj" 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.026 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker__:inst__myobj 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst__myobj", 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.027 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.028 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker__ns:inst__myobj 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst__myobj", 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.029 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj", 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.030 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker__ns:inst__myobj 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker___myobj_", 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.031 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.032 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker___myobj_ 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker___myobj_", 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.033 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker___myobj_ 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.034 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns__myobj_", 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.035 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker__ns__myobj_ 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns__myobj_", 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.036 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.037 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "", 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker__ns__myobj_ 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst__myobj_", 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.038 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.039 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker__:inst__myobj_ 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__:inst__myobj_", 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.040 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "" 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker__:inst__myobj_ 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj": { 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst__myobj_", 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj": { 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.042 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout:raw=.data-pool:marker__ns:inst__myobj_ 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: "raw_obj2": { 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: "pool": ".data-pool", 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: "oid": "marker__ns:inst__myobj_", 2024-03-04T17:52:59.043 INFO:tasks.workunit.client.0.smithi047.stdout: "loc": "" 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "new_obj2": { 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket": { 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "ebtest", 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "marker": "marker", 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "tenant": "", 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "explicit_placement": { 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "data_extra_pool": "", 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.044 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout: "key": { 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout: "name": "_myobj_", 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout: "instance": "inst", 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout: "ns": "ns" 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout:raw2=.data-pool:marker__ns:inst__myobj_ 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] TestRGWObj.old_to_raw (1 ms) 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 4 tests from TestRGWObj (2 ms total) 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 4 tests from 1 test suite ran. (2 ms total) 2024-03-04T17:52:59.045 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 4 tests. 2024-03-04T17:52:59.045 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:52:59.046 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:52:59.060 INFO:tasks.workunit:Running workunits matching rgw/test_librgw_file.sh on client.0... 2024-03-04T17:52:59.061 INFO:tasks.workunit:Running workunit rgw/test_librgw_file.sh... 2024-03-04T17:52:59.061 DEBUG:teuthology.orchestra.run.smithi047:workunit test rgw/test_librgw_file.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh 2024-03-04T17:52:59.168 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:59.169 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:59.225 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.231+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.231+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.231+0000 7f4cb7b81980 20 realm 2024-03-04T17:52:59.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.231+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.231+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.231+0000 7f4cb7b81980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:52:59.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.231+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.231+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.231+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.228 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.235+0000 7f4cb7b81980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:52:59.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.235+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.235+0000 7f4cb7b81980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:52:59.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.235+0000 7f4cb7b81980 20 searching for the correct realm 2024-03-04T17:52:59.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.255+0000 7f4cb4e98640 0 -- 172.21.15.47:0/1211362379 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f4cac0fec10 msgr2=0x7f4cac009940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:52:59.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.475+0000 7f4cb4e98640 0 -- 172.21.15.47:0/1211362379 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55b12dd7d4b0 msgr2=0x55b12dd7f950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:52:59.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.487+0000 7f4cb7b81980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:52:59.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.487+0000 7f4cb7b81980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:52:59.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.487+0000 7f4cb7b81980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:52:59.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.487+0000 7f4cb7b81980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:52:59.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.487+0000 7f4cb7b81980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:52:59.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.487+0000 7f4cb7b81980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:52:59.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.487+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.487+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.487+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:52:59.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:52:59.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 zone default found 2024-03-04T17:52:59.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 4 Realm: () 2024-03-04T17:52:59.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:52:59.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:52:59.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:52:59.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 zonegroup default 2024-03-04T17:52:59.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.491+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.495+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.495+0000 7f4cb7b81980 20 started sync module instance, tier type = 2024-03-04T17:52:59.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.495+0000 7f4cb7b81980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:52:59.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.515+0000 7f4cb7b81980 20 add_watcher() i=0 2024-03-04T17:52:59.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.515+0000 7f4cb7b81980 20 add_watcher() i=1 2024-03-04T17:52:59.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.515+0000 7f4cb7b81980 20 add_watcher() i=2 2024-03-04T17:52:59.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.515+0000 7f4cb7b81980 20 add_watcher() i=3 2024-03-04T17:52:59.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.515+0000 7f4cb7b81980 20 add_watcher() i=4 2024-03-04T17:52:59.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.515+0000 7f4cb7b81980 20 add_watcher() i=5 2024-03-04T17:52:59.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.515+0000 7f4cb7b81980 20 add_watcher() i=6 2024-03-04T17:52:59.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.519+0000 7f4cb7b81980 20 add_watcher() i=7 2024-03-04T17:52:59.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.519+0000 7f4cb7b81980 2 all 8 watchers are set, enabling cache 2024-03-04T17:52:59.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4b6bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:52:59.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4cb7b81980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:52:59.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4cb7b81980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:52:59.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4cb7b81980 5 note: GC not initialized 2024-03-04T17:52:59.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4b6afe5640 20 reqs_thread_entry: start 2024-03-04T17:52:59.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4cb7b81980 20 init_complete bucket index max shards: 11 2024-03-04T17:52:59.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4cb7b81980 20 Filter name: none 2024-03-04T17:52:59.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4b68fe1640 20 reqs_thread_entry: start 2024-03-04T17:52:59.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4cb7b81980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-librgw-file : miss 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.523+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x0 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 adding default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 cache get: name=default.rgw.meta+users.email+librgw@example.com : miss 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x0 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 adding default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-03-04T17:52:59.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 cache get: name=default.rgw.meta+users.keys+mB8jPtera7o+w177Kp0ZK6fNd4w= : miss 2024-03-04T17:52:59.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 20 rados->read ofs=0 len=0 2024-03-04T17:52:59.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:59.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 cache put: name=default.rgw.meta+users.keys+mB8jPtera7o+w177Kp0ZK6fNd4w= info.flags=0x0 2024-03-04T17:52:59.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 adding default.rgw.meta+users.keys+mB8jPtera7o+w177Kp0ZK6fNd4w= to cache LRU end 2024-03-04T17:52:59.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 cache get: name=default.rgw.meta+users.keys+mB8jPtera7o+w177Kp0ZK6fNd4w= : hit (negative entry) 2024-03-04T17:52:59.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 cache get: name=default.rgw.meta+users.keys+mB8jPtera7o+w177Kp0ZK6fNd4w= : hit (negative entry) 2024-03-04T17:52:59.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-03-04T17:52:59.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-03-04T17:52:59.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.527+0000 7f4cb7b81980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-librgw-file, ofs0, ns] 2024-03-04T17:52:59.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.531+0000 7f4b98ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1065151889409 cookie 94219466742128 notifier 5243 bl.length()=664 2024-03-04T17:52:59.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.531+0000 7f4b98ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-03-04T17:52:59.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.531+0000 7f4b98ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-03-04T17:52:59.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.535+0000 7f4cb7b81980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-03-04T17:52:59.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.535+0000 7f4cb7b81980 10 moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-03-04T17:52:59.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.535+0000 7f4cb7b81980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:librgw@example.com, ofs0, ns] 2024-03-04T17:52:59.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.535+0000 7f4b98ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1065151889408 cookie 94219466698688 notifier 5243 bl.length()=194 2024-03-04T17:52:59.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.535+0000 7f4b98ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-03-04T17:52:59.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.535+0000 7f4b98ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-03-04T17:52:59.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.543+0000 7f4cb7b81980 10 cache put: name=default.rgw.meta+users.keys+mB8jPtera7o+w177Kp0ZK6fNd4w= info.flags=0x7 2024-03-04T17:52:59.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.543+0000 7f4cb7b81980 10 moving default.rgw.meta+users.keys+mB8jPtera7o+w177Kp0ZK6fNd4w= to cache LRU end 2024-03-04T17:52:59.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.543+0000 7f4cb7b81980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:mB8jPtera7o+w177Kp0ZK6fNd4w=, ofs0, ns] 2024-03-04T17:52:59.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.543+0000 7f4b87fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1065151889409 cookie 94219466698688 notifier 5243 bl.length()=203 2024-03-04T17:52:59.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.543+0000 7f4b87fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+mB8jPtera7o+w177Kp0ZK6fNd4w= info.flags=0x7 2024-03-04T17:52:59.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.543+0000 7f4b87fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+mB8jPtera7o+w177Kp0ZK6fNd4w= to cache LRU end 2024-03-04T17:52:59.540 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:52:59.540 INFO:tasks.workunit.client.0.smithi047.stdout: "user_id": "ceph-test-librgw-file", 2024-03-04T17:52:59.540 INFO:tasks.workunit.client.0.smithi047.stdout: "display_name": "librgw test user", 2024-03-04T17:52:59.540 INFO:tasks.workunit.client.0.smithi047.stdout: "email": "librgw@example.com", 2024-03-04T17:52:59.540 INFO:tasks.workunit.client.0.smithi047.stdout: "suspended": 0, 2024-03-04T17:52:59.540 INFO:tasks.workunit.client.0.smithi047.stdout: "max_buckets": 1000, 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "subusers": [], 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "keys": [ 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: { 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "user": "ceph-test-librgw-file", 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "access_key": "mB8jPtera7o+w177Kp0ZK6fNd4w=", 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "secret_key": "K+VqYxdzwAqDA/a6ssc86nm4G7CqjsIztsK9kcdap8bUtB7oMc2i+A==", 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "active": true 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: ], 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "swift_keys": [], 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "caps": [], 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "default_placement": "", 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "default_storage_class": "", 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "placement_tags": [], 2024-03-04T17:52:59.541 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_quota": { 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "enabled": false, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size": -1, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "max_objects": -1 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "user_quota": { 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "enabled": false, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size": -1, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "max_objects": -1 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "temp_url_keys": [], 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "type": "rgw", 2024-03-04T17:52:59.542 INFO:tasks.workunit.client.0.smithi047.stdout: "mfa_ids": [] 2024-03-04T17:52:59.543 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:52:59.543 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:52:59.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.547+0000 7f4b6bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:52:59.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.551+0000 7f4cb7b81980 20 remove_watcher() i=0 2024-03-04T17:52:59.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.551+0000 7f4cb7b81980 2 removed watcher, disabling cache 2024-03-04T17:52:59.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.555+0000 7f4cb7b81980 20 remove_watcher() i=1 2024-03-04T17:52:59.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.555+0000 7f4cb7b81980 20 remove_watcher() i=2 2024-03-04T17:52:59.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.559+0000 7f4cb7b81980 20 remove_watcher() i=3 2024-03-04T17:52:59.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.559+0000 7f4cb7b81980 20 remove_watcher() i=4 2024-03-04T17:52:59.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.563+0000 7f4cb7b81980 20 remove_watcher() i=5 2024-03-04T17:52:59.560 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.563+0000 7f4cb7b81980 20 remove_watcher() i=6 2024-03-04T17:52:59.563 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:52:59.567+0000 7f4cb7b81980 20 remove_watcher() i=7 2024-03-04T17:52:59.579 INFO:tasks.workunit.client.0.smithi047.stdout:phase 1.1 2024-03-04T17:52:59.597 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 24 tests from 1 test suite. 2024-03-04T17:52:59.597 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:59.597 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 24 tests from LibRGW 2024-03-04T17:52:59.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:59.610 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:59.610 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:59.908 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (311 ms) 2024-03-04T17:52:59.908 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-03-04T17:52:59.910 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-03-04T17:52:59.910 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:59.910 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-03-04T17:52:59.910 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-03-04T17:52:59.932 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/baz/quux 2024-03-04T17:52:59.941 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/f1 2024-03-04T17:52:59.947 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/f2 2024-03-04T17:52:59.953 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/f1 2024-03-04T17:52:59.959 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/d1/ 2024-03-04T17:52:59.966 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/baz/hungry 2024-03-04T17:52:59.973 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/baz/hungry/ 2024-03-04T17:52:59.979 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/baz/momma 2024-03-04T17:52:59.984 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/baz/bear/ 2024-03-04T17:52:59.991 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/baz/sasquatch 2024-03-04T17:52:59.997 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/baz/sasquatch/ 2024-03-04T17:53:00.005 INFO:tasks.workunit.client.0.smithi047.stdout:creating: nfsroot:foo/bar/baz/frobozz 2024-03-04T17:53:00.013 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SETUP_HIER1 (103 ms) 2024-03-04T17:53:00.013 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-03-04T17:53:00.246 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SETUP_DIRS1 (233 ms) 2024-03-04T17:53:00.247 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SETATTR 2024-03-04T17:53:00.247 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-03-04T17:53:00.247 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-03-04T17:53:00.295 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (49 ms) 2024-03-04T17:53:00.296 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-03-04T17:53:00.396 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (101 ms) 2024-03-04T17:53:00.397 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-03-04T17:53:00.397 INFO:tasks.workunit.client.0.smithi047.stdout:rename file rfile_0 to rfile_r0 (bucket brename0) 2024-03-04T17:53:00.413 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (17 ms) 2024-03-04T17:53:00.413 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-03-04T17:53:00.413 INFO:tasks.workunit.client.0.smithi047.stdout:rename file rfile_1 (bucket brename0) to rfile_rhilldog (bucket brename1) 2024-03-04T17:53:00.430 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (17 ms) 2024-03-04T17:53:00.430 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout:scanning objects in /bdirs1/dir_0 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout:scanning objects in /bdirs1/dir_1 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout:scanning objects in /bdirs1/dir_2 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:53:00.431 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:00.432 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:53:00.432 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-03-04T17:53:00.432 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-03-04T17:53:00.432 INFO:tasks.workunit.client.0.smithi047.stdout:read back objects in /bdirs1/dir_0 2024-03-04T17:53:00.432 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_0/sfile_0 2024-03-04T17:53:00.434 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:53:00.434 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_0/sfile_1 2024-03-04T17:53:00.437 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:53:00.437 INFO:tasks.workunit.client.0.smithi047.stdout:read back objects in /bdirs1/dir_1 2024-03-04T17:53:00.437 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_1/sfile_0 2024-03-04T17:53:00.439 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:53:00.439 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_1/sfile_1 2024-03-04T17:53:00.443 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:53:00.443 INFO:tasks.workunit.client.0.smithi047.stdout:read back objects in /bdirs1/dir_2 2024-03-04T17:53:00.443 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_2/sfile_0 2024-03-04T17:53:00.445 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:53:00.445 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_2/sfile_1 2024-03-04T17:53:00.449 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:53:00.449 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READ_DIRS1 (19 ms) 2024-03-04T17:53:00.449 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-03-04T17:53:00.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READF_SETUP1 (230 ms) 2024-03-04T17:53:00.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:53:00.680 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:53:00.681 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:53:00.681 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:53:00.681 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:53:00.681 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-03-04T17:53:00.681 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.HIER1 2024-03-04T17:53:00.696 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/f2 (f2): FILE 2024-03-04T17:53:00.696 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/f1 (f1): FILE 2024-03-04T17:53:00.702 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-03-04T17:53:00.706 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-03-04T17:53:00.713 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-03-04T17:53:00.714 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-03-04T17:53:00.715 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-03-04T17:53:00.716 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-03-04T17:53:00.716 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-03-04T17:53:00.717 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-03-04T17:53:00.718 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-03-04T17:53:00.723 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-03-04T17:53:00.723 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-03-04T17:53:00.723 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar (bar): DIR 2024-03-04T17:53:00.723 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo (foo): DIR 2024-03-04T17:53:00.723 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot (nfsroot): DIR 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.HIER1 (43 ms) 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:53:00.724 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:00.725 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:00.725 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:00.725 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:00.725 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:00.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:00.727+0000 7fdab8a73a80 -1 shutting down 2024-03-04T17:53:00.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:00.727+0000 7fd9945e6640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 54044, si_uid : 1000, si_addr0x3e80000d31c, si_status0 2024-03-04T17:53:00.865 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (143 ms) 2024-03-04T17:53:00.866 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 24 tests from LibRGW (1268 ms total) 2024-03-04T17:53:00.866 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:00.866 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:00.866 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 24 tests from 1 test suite ran. (1268 ms total) 2024-03-04T17:53:00.866 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 24 tests. 2024-03-04T17:53:00.882 INFO:tasks.workunit.client.0.smithi047.stdout:phase 1.2 2024-03-04T17:53:00.903 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 24 tests from 1 test suite. 2024-03-04T17:53:00.903 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:00.903 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 24 tests from LibRGW 2024-03-04T17:53:00.904 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:00.914 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:00.914 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:01.675 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (771 ms) 2024-03-04T17:53:01.675 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-03-04T17:53:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT_NOROOT (1 ms) 2024-03-04T17:53:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-03-04T17:53:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-03-04T17:53:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SETUP_HIER1 (0 ms) 2024-03-04T17:53:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-03-04T17:53:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SETUP_DIRS1 (0 ms) 2024-03-04T17:53:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SETATTR 2024-03-04T17:53:01.677 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READ_DIRS1 (0 ms) 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.HIER1 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.HIER1 (0 ms) 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:53:01.679 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-03-04T17:53:01.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:53:01.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-03-04T17:53:01.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:53:01.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:53:01.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:01.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:01.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:01.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:01.680 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:01.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:01.679+0000 7fe0f68fda80 -1 shutting down 2024-03-04T17:53:01.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:01.683+0000 7fdfd2470640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 54687, si_uid : 1000, si_addr0x3e80000d59f, si_status0 2024-03-04T17:53:01.830 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (153 ms) 2024-03-04T17:53:01.830 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 24 tests from LibRGW (926 ms total) 2024-03-04T17:53:01.830 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:01.830 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:01.830 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 24 tests from 1 test suite ran. (926 ms total) 2024-03-04T17:53:01.830 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 24 tests. 2024-03-04T17:53:01.844 INFO:tasks.workunit.client.0.smithi047.stdout:phase 1.3 2024-03-04T17:53:01.866 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 24 tests from 1 test suite. 2024-03-04T17:53:01.866 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:01.866 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 24 tests from LibRGW 2024-03-04T17:53:01.866 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:01.879 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:01.879 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:02.151 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (285 ms) 2024-03-04T17:53:02.151 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-03-04T17:53:02.153 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-03-04T17:53:02.153 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:02.153 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-03-04T17:53:02.153 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-03-04T17:53:02.165 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SETUP_HIER1 (12 ms) 2024-03-04T17:53:02.165 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-03-04T17:53:02.281 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SETUP_DIRS1 (116 ms) 2024-03-04T17:53:02.281 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SETATTR 2024-03-04T17:53:02.281 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-03-04T17:53:02.281 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:scanning objects in /bdirs1/dir_0 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout:scanning objects in /bdirs1/dir_1 2024-03-04T17:53:02.282 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout:scanning objects in /bdirs1/dir_2 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout:read back objects in /bdirs1/dir_0 2024-03-04T17:53:02.283 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_0/sfile_0 2024-03-04T17:53:02.284 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:53:02.284 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_0/sfile_1 2024-03-04T17:53:02.286 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:53:02.286 INFO:tasks.workunit.client.0.smithi047.stdout:read back objects in /bdirs1/dir_1 2024-03-04T17:53:02.287 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_1/sfile_0 2024-03-04T17:53:02.289 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:53:02.289 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_1/sfile_1 2024-03-04T17:53:02.292 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:53:02.292 INFO:tasks.workunit.client.0.smithi047.stdout:read back objects in /bdirs1/dir_2 2024-03-04T17:53:02.292 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_2/sfile_0 2024-03-04T17:53:02.294 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:53:02.294 INFO:tasks.workunit.client.0.smithi047.stdout:reading 0,256 dir_2/sfile_1 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READ_DIRS1 (16 ms) 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-03-04T17:53:02.298 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (1 ms) 2024-03-04T17:53:02.299 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.HIER1 2024-03-04T17:53:02.316 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/f2 (f2): FILE 2024-03-04T17:53:02.318 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/f1 (f1): FILE 2024-03-04T17:53:02.324 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-03-04T17:53:02.328 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-03-04T17:53:02.336 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-03-04T17:53:02.336 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-03-04T17:53:02.338 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-03-04T17:53:02.338 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-03-04T17:53:02.340 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-03-04T17:53:02.341 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-03-04T17:53:02.342 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-03-04T17:53:02.346 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-03-04T17:53:02.346 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo/bar (bar): DIR 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot/foo (foo): DIR 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:/nfsroot (nfsroot): DIR 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.HIER1 (48 ms) 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:53:02.347 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:02.348 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:02.348 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:02.348 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:02.348 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:02.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:02.351+0000 7f70893eea80 -1 shutting down 2024-03-04T17:53:02.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:02.351+0000 7f6f64f61640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 55330, si_uid : 1000, si_addr0x3e80000d822, si_status0 2024-03-04T17:53:02.522 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (175 ms) 2024-03-04T17:53:02.522 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 24 tests from LibRGW (655 ms total) 2024-03-04T17:53:02.522 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:02.522 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:02.522 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 24 tests from 1 test suite ran. (655 ms total) 2024-03-04T17:53:02.522 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 24 tests. 2024-03-04T17:53:02.536 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2.1 2024-03-04T17:53:02.558 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 9 tests from 1 test suite. 2024-03-04T17:53:02.558 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:02.558 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 9 tests from LibRGW 2024-03-04T17:53:02.558 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:02.570 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:02.570 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:02.866 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (308 ms) 2024-03-04T17:53:02.866 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:02.868 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-03-04T17:53:02.868 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:53:02.889 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CREATE_BUCKET (21 ms) 2024-03-04T17:53:02.889 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:53:02.889 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-03-04T17:53:02.890 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-03-04T17:53:02.910 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave0 2024-03-04T17:53:02.928 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave1 2024-03-04T17:53:02.946 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave2 2024-03-04T17:53:02.968 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave3 2024-03-04T17:53:02.989 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave4 2024-03-04T17:53:03.011 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave5 2024-03-04T17:53:03.031 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave6 2024-03-04T17:53:03.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave7 2024-03-04T17:53:03.076 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave8 2024-03-04T17:53:03.101 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave9 2024-03-04T17:53:03.101 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (211 ms) 2024-03-04T17:53:03.101 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-03-04T17:53:03.833 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (733 ms) 2024-03-04T17:53:03.833 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:03.833 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:03.833 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:03.833 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:03.834 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:03.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:03.839+0000 7efe33211a80 -1 shutting down 2024-03-04T17:53:03.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:03.839+0000 7efd0ed84640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 55973, si_uid : 1000, si_addr0x3e80000daa5, si_status0 2024-03-04T17:53:03.917 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (83 ms) 2024-03-04T17:53:03.917 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 9 tests from LibRGW (1358 ms total) 2024-03-04T17:53:03.917 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:03.917 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:03.917 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 9 tests from 1 test suite ran. (1359 ms total) 2024-03-04T17:53:03.917 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 9 tests. 2024-03-04T17:53:03.925 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2.2 2024-03-04T17:53:03.946 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 9 tests from 1 test suite. 2024-03-04T17:53:03.946 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:03.946 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 9 tests from LibRGW 2024-03-04T17:53:03.946 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:03.957 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:03.957 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:04.229 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (283 ms) 2024-03-04T17:53:04.230 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:04.231 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-03-04T17:53:04.231 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:53:04.231 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-03-04T17:53:04.231 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:53:04.288 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_BUCKET (57 ms) 2024-03-04T17:53:04.288 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-03-04T17:53:04.305 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave0 2024-03-04T17:53:04.321 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave1 2024-03-04T17:53:04.342 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave2 2024-03-04T17:53:04.362 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave3 2024-03-04T17:53:04.383 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave4 2024-03-04T17:53:04.402 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave5 2024-03-04T17:53:04.420 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave6 2024-03-04T17:53:04.437 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave7 2024-03-04T17:53:04.455 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave8 2024-03-04T17:53:04.475 INFO:tasks.workunit.client.0.smithi047.stdout:created: sorrydave9 2024-03-04T17:53:04.475 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (187 ms) 2024-03-04T17:53:04.475 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-03-04T17:53:05.288 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (812 ms) 2024-03-04T17:53:05.288 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:05.288 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:05.288 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:05.288 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:05.288 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:05.288 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:05.291+0000 7f61d60daa80 -1 shutting down 2024-03-04T17:53:05.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:05.295+0000 7f60b1c4d640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 56616, si_uid : 1000, si_addr0x3e80000dd28, si_status0 2024-03-04T17:53:05.381 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (94 ms) 2024-03-04T17:53:05.381 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 9 tests from LibRGW (1435 ms total) 2024-03-04T17:53:05.381 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:05.381 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:05.381 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 9 tests from 1 test suite ran. (1435 ms total) 2024-03-04T17:53:05.382 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 9 tests. 2024-03-04T17:53:05.392 INFO:tasks.workunit.client.0.smithi047.stdout:phase 3.1 2024-03-04T17:53:05.737 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 13 tests from 1 test suite. 2024-03-04T17:53:05.737 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:05.737 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 13 tests from LibRGW 2024-03-04T17:53:05.737 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:05.746 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:05.746 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:06.076 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (339 ms) 2024-03-04T17:53:06.076 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:06.077 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:53:06.077 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:53:06.100 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CREATE_BUCKET (22 ms) 2024-03-04T17:53:06.100 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-03-04T17:53:06.108 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (9 ms) 2024-03-04T17:53:06.108 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-03-04T17:53:06.112 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (4 ms) 2024-03-04T17:53:06.112 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LARGE1 2024-03-04T17:53:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LARGE1 (79 ms) 2024-03-04T17:53:06.191 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LARGE2 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LARGE2 (62 ms) 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:06.254 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:06.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:06.259+0000 7f03c7822a80 -1 shutting down 2024-03-04T17:53:06.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:06.259+0000 7f02a3395640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57259, si_uid : 1000, si_addr0x3e80000dfab, si_status0 2024-03-04T17:53:06.439 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (186 ms) 2024-03-04T17:53:06.439 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 13 tests from LibRGW (703 ms total) 2024-03-04T17:53:06.439 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:06.439 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:06.439 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 13 tests from 1 test suite ran. (703 ms total) 2024-03-04T17:53:06.439 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 13 tests. 2024-03-04T17:53:06.457 INFO:tasks.workunit.client.0.smithi047.stdout:phase 3.2 2024-03-04T17:53:06.802 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 13 tests from 1 test suite. 2024-03-04T17:53:06.802 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:06.802 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 13 tests from LibRGW 2024-03-04T17:53:06.802 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:06.811 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:06.811 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:07.097 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (295 ms) 2024-03-04T17:53:07.098 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:07.099 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-03-04T17:53:07.099 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:53:07.099 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-03-04T17:53:07.099 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-03-04T17:53:07.111 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (12 ms) 2024-03-04T17:53:07.112 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-03-04T17:53:07.119 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (8 ms) 2024-03-04T17:53:07.119 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LARGE1 2024-03-04T17:53:07.661 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LARGE1 (542 ms) 2024-03-04T17:53:07.661 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LARGE2 2024-03-04T17:53:07.662 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LARGE2 (0 ms) 2024-03-04T17:53:07.662 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-03-04T17:53:07.662 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-03-04T17:53:07.662 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-03-04T17:53:07.672 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_OBJECT (11 ms) 2024-03-04T17:53:07.672 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:53:07.754 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_BUCKET (81 ms) 2024-03-04T17:53:07.754 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:07.754 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:07.754 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:07.754 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:07.754 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:07.755 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:07.759+0000 7f06425bca80 -1 shutting down 2024-03-04T17:53:07.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:07.759+0000 7f051e12f640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57902, si_uid : 1000, si_addr0x3e80000e22e, si_status0 2024-03-04T17:53:07.856 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (102 ms) 2024-03-04T17:53:07.856 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 13 tests from LibRGW (1053 ms total) 2024-03-04T17:53:07.856 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:07.856 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:07.856 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 13 tests from 1 test suite ran. (1053 ms total) 2024-03-04T17:53:07.856 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 13 tests. 2024-03-04T17:53:07.872 INFO:tasks.workunit.client.0.smithi047.stdout:phase 4.1 2024-03-04T17:53:07.893 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 11 tests from 1 test suite. 2024-03-04T17:53:07.893 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:07.893 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 11 tests from LibRGW 2024-03-04T17:53:07.893 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.TVAR 2024-03-04T17:53:07.893 INFO:tasks.workunit.client.0.smithi047.stdout:read i1: 64001 2024-03-04T17:53:07.893 INFO:tasks.workunit.client.0.smithi047.stdout:read s1: blunderbuss 2024-03-04T17:53:07.893 INFO:tasks.workunit.client.0.smithi047.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-03-04T17:53:07.894 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:07.907 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:07.907 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:08.696 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (803 ms) 2024-03-04T17:53:08.696 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:08.698 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-03-04T17:53:08.698 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:53:08.738 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (39 ms) 2024-03-04T17:53:08.738 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:53:10.048 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_0 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_1 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_2 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_3 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_4 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_5 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_6 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_7 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_8 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_9 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_10 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_11 2024-03-04T17:53:10.049 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_12 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_13 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_14 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_15 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_16 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_17 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_18 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_19 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_20 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_21 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_22 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_23 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_24 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_25 2024-03-04T17:53:10.050 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_26 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_27 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_28 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_29 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_30 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_31 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_32 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_33 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_34 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_35 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_36 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_37 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_38 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_39 2024-03-04T17:53:10.051 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_40 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_41 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_42 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_43 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_44 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_45 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_46 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_47 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_48 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_49 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_50 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_51 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_52 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_53 2024-03-04T17:53:10.052 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_54 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_55 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_56 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_57 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_58 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_59 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_60 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_61 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_62 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_63 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_64 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_65 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_66 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_67 2024-03-04T17:53:10.053 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_68 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_69 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_70 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_71 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_72 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_73 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_74 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_75 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_76 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_77 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_78 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_79 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_80 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_81 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_82 2024-03-04T17:53:10.054 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_83 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_84 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_85 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_86 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_87 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_88 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_89 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_90 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_91 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_92 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_93 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_94 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_95 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_96 2024-03-04T17:53:10.055 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_97 2024-03-04T17:53:10.056 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_98 2024-03-04T17:53:10.056 INFO:tasks.workunit.client.0.smithi047.stdout:created: dmarker:f_99 2024-03-04T17:53:10.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (1310 ms) 2024-03-04T17:53:10.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:53:10.056 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=. flags=4 2024-03-04T17:53:10.056 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=.. flags=4 2024-03-04T17:53:10.056 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-03-04T17:53:10.057 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-03-04T17:53:10.058 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-03-04T17:53:10.059 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-03-04T17:53:10.060 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-03-04T17:53:10.061 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-03-04T17:53:10.062 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_READDIR (7 ms) 2024-03-04T17:53:10.063 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-03-04T17:53:10.064 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-03-04T17:53:10.064 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-03-04T17:53:10.064 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-03-04T17:53:10.065 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-03-04T17:53:10.066 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-03-04T17:53:10.067 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-03-04T17:53:10.068 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-03-04T17:53:10.069 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-03-04T17:53:10.070 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER2_READDIR (3 ms) 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:10.071 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (1 ms) 2024-03-04T17:53:10.072 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:10.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:10.063+0000 7f32b66e1a80 -1 shutting down 2024-03-04T17:53:10.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:10.067+0000 7f3192254640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 58545, si_uid : 1000, si_addr0x3e80000e4b1, si_status0 2024-03-04T17:53:10.196 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (136 ms) 2024-03-04T17:53:10.196 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 11 tests from LibRGW (2302 ms total) 2024-03-04T17:53:10.196 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:10.196 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:10.196 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 11 tests from 1 test suite ran. (2302 ms total) 2024-03-04T17:53:10.197 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 11 tests. 2024-03-04T17:53:10.211 INFO:tasks.workunit.client.0.smithi047.stdout:phase 4.2 2024-03-04T17:53:10.234 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 11 tests from 1 test suite. 2024-03-04T17:53:10.234 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:10.234 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 11 tests from LibRGW 2024-03-04T17:53:10.234 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.TVAR 2024-03-04T17:53:10.234 INFO:tasks.workunit.client.0.smithi047.stdout:read i1: 64001 2024-03-04T17:53:10.234 INFO:tasks.workunit.client.0.smithi047.stdout:read s1: blunderbuss 2024-03-04T17:53:10.234 INFO:tasks.workunit.client.0.smithi047.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-03-04T17:53:10.234 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:10.244 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:10.244 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:10.771 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (537 ms) 2024-03-04T17:53:10.771 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:10.772 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:53:10.772 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:53:10.789 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (17 ms) 2024-03-04T17:53:10.790 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:53:10.790 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-03-04T17:53:10.790 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:53:10.790 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-03-04T17:53:10.790 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-03-04T17:53:10.793 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-03-04T17:53:10.793 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-03-04T17:53:10.793 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-03-04T17:53:10.794 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-03-04T17:53:10.795 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-03-04T17:53:10.796 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-03-04T17:53:10.797 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-03-04T17:53:10.798 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-03-04T17:53:10.799 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-03-04T17:53:10.800 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER2_READDIR (4 ms) 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:10.801 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:10.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:10.799+0000 7f79767ffa80 -1 shutting down 2024-03-04T17:53:10.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:10.799+0000 7f7852372640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 59188, si_uid : 1000, si_addr0x3e80000e734, si_status0 2024-03-04T17:53:10.931 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (137 ms) 2024-03-04T17:53:10.931 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 11 tests from LibRGW (696 ms total) 2024-03-04T17:53:10.931 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:10.931 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:10.931 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 11 tests from 1 test suite ran. (696 ms total) 2024-03-04T17:53:10.931 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 11 tests. 2024-03-04T17:53:10.945 INFO:tasks.workunit.client.0.smithi047.stdout:phase 5.1 2024-03-04T17:53:10.979 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 19 tests from 1 test suite. 2024-03-04T17:53:10.979 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:10.979 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 19 tests from LibRGW 2024-03-04T17:53:10.979 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:10.990 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:10.990 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:11.552 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (574 ms) 2024-03-04T17:53:11.552 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:11.553 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:53:11.554 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:53:11.574 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CREATE_BUCKET (21 ms) 2024-03-04T17:53:11.574 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-03-04T17:53:11.581 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (7 ms) 2024-03-04T17:53:11.581 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-03-04T17:53:11.581 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-03-04T17:53:11.581 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-03-04T17:53:11.585 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (4 ms) 2024-03-04T17:53:11.585 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-03-04T17:53:11.585 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-03-04T17:53:11.585 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-03-04T17:53:11.592 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.PUT_OBJECT (7 ms) 2024-03-04T17:53:11.592 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-03-04T17:53:11.596 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.GET_OBJECT (3 ms) 2024-03-04T17:53:11.596 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-03-04T17:53:11.596 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-03-04T17:53:11.596 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-03-04T17:53:11.596 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-03-04T17:53:11.596 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.WRITEV 2024-03-04T17:53:11.596 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-03-04T17:53:11.596 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READV 2024-03-04T17:53:11.596 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READV (0 ms) 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:11.597 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:11.599+0000 7fe3fe3d7a80 -1 shutting down 2024-03-04T17:53:11.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:11.603+0000 7fe2d9f4a640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 59831, si_uid : 1000, si_addr0x3e80000e9b7, si_status0 2024-03-04T17:53:11.698 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (103 ms) 2024-03-04T17:53:11.698 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 19 tests from LibRGW (720 ms total) 2024-03-04T17:53:11.698 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:11.699 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:11.699 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 19 tests from 1 test suite ran. (720 ms total) 2024-03-04T17:53:11.699 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 19 tests. 2024-03-04T17:53:11.712 INFO:tasks.workunit.client.0.smithi047.stdout:phase 5.2 2024-03-04T17:53:11.746 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] Running 19 tests from 1 test suite. 2024-03-04T17:53:11.746 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment set-up. 2024-03-04T17:53:11.746 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 19 tests from LibRGW 2024-03-04T17:53:11.746 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:53:11.755 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:11.755 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:12.045 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.INIT (299 ms) 2024-03-04T17:53:12.045 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:53:12.047 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-03-04T17:53:12.047 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:53:12.047 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-03-04T17:53:12.047 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-03-04T17:53:12.055 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (8 ms) 2024-03-04T17:53:12.055 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-03-04T17:53:12.055 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (0 ms) 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.PUT_OBJECT (0 ms) 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.GET_OBJECT (0 ms) 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.WRITEV 2024-03-04T17:53:12.056 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-03-04T17:53:12.057 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READV 2024-03-04T17:53:12.057 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READV (0 ms) 2024-03-04T17:53:12.057 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-03-04T17:53:12.057 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-03-04T17:53:12.057 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-03-04T17:53:12.062 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_OBJECT (7 ms) 2024-03-04T17:53:12.062 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:53:12.119 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.DELETE_BUCKET (57 ms) 2024-03-04T17:53:12.120 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:53:12.120 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:53:12.120 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:53:12.120 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:53:12.120 INFO:tasks.workunit.client.0.smithi047.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:53:12.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.123+0000 7f6c8b012a80 -1 shutting down 2024-03-04T17:53:12.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.127+0000 7f6b66b85640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60474, si_uid : 1000, si_addr0x3e80000ec3a, si_status0 2024-03-04T17:53:12.191 INFO:tasks.workunit.client.0.smithi047.stdout:[ OK ] LibRGW.SHUTDOWN (72 ms) 2024-03-04T17:53:12.191 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] 19 tests from LibRGW (445 ms total) 2024-03-04T17:53:12.191 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:12.191 INFO:tasks.workunit.client.0.smithi047.stdout:[----------] Global test environment tear-down 2024-03-04T17:53:12.191 INFO:tasks.workunit.client.0.smithi047.stdout:[==========] 19 tests from 1 test suite ran. (445 ms total) 2024-03-04T17:53:12.191 INFO:tasks.workunit.client.0.smithi047.stdout:[ PASSED ] 19 tests. 2024-03-04T17:53:12.198 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:53:12.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:53:12.210 INFO:tasks.workunit:Running workunits matching rgw/test_awssdkv4_sig.sh on client.0... 2024-03-04T17:53:12.211 INFO:tasks.workunit:Running workunit rgw/test_awssdkv4_sig.sh... 2024-03-04T17:53:12.211 DEBUG:teuthology.orchestra.run.smithi047:workunit test rgw/test_awssdkv4_sig.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_awssdkv4_sig.sh 2024-03-04T17:53:12.265 INFO:tasks.workunit.client.0.smithi047.stderr:+ [ -z ] 2024-03-04T17:53:12.265 INFO:tasks.workunit.client.0.smithi047.stderr:+ export AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-03-04T17:53:12.265 INFO:tasks.workunit.client.0.smithi047.stderr:+ export AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-03-04T17:53:12.265 INFO:tasks.workunit.client.0.smithi047.stderr:+ radosgw-admin user create --uid ceph-test-maven --access-key lNCnR47C2g+ZidCWBAUuwfSAA7Q= --secret tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== --display-name maven test user --email sigv4@example.com 2024-03-04T17:53:12.308 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:53:12.308 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:53:12.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.359+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.359+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.359+0000 7fc7d11b5980 20 realm 2024-03-04T17:53:12.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.359+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.359+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.359+0000 7fc7d11b5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:53:12.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.359+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.359+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.359+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.363+0000 7fc7d11b5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:53:12.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.363+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.363+0000 7fc7d11b5980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:53:12.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.363+0000 7fc7d11b5980 20 searching for the correct realm 2024-03-04T17:53:12.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.367+0000 7fc7c64cc640 0 -- 172.21.15.47:0/788115190 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fc7b80df050 msgr2=0x7fc7b81276f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:53:12.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.375+0000 7fc7cf4ce640 0 -- 172.21.15.47:0/788115190 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x563a82508050 msgr2=0x563a8250a440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:53:12.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:53:12.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:53:12.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:53:12.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:53:12.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:53:12.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:53:12.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 zone default found 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 4 Realm: () 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 zonegroup default 2024-03-04T17:53:12.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.383+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.387+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.387+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.387+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.387+0000 7fc7d11b5980 20 started sync module instance, tier type = 2024-03-04T17:53:12.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.387+0000 7fc7d11b5980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:53:12.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.403+0000 7fc7d11b5980 20 add_watcher() i=0 2024-03-04T17:53:12.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.403+0000 7fc7d11b5980 20 add_watcher() i=1 2024-03-04T17:53:12.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.403+0000 7fc7d11b5980 20 add_watcher() i=2 2024-03-04T17:53:12.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.403+0000 7fc7d11b5980 20 add_watcher() i=3 2024-03-04T17:53:12.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.403+0000 7fc7d11b5980 20 add_watcher() i=4 2024-03-04T17:53:12.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.403+0000 7fc7d11b5980 20 add_watcher() i=5 2024-03-04T17:53:12.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.403+0000 7fc7d11b5980 20 add_watcher() i=6 2024-03-04T17:53:12.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.407+0000 7fc7d11b5980 20 add_watcher() i=7 2024-03-04T17:53:12.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.407+0000 7fc7d11b5980 2 all 8 watchers are set, enabling cache 2024-03-04T17:53:12.404 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc6857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:53:12.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:53:12.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:53:12.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 5 note: GC not initialized 2024-03-04T17:53:12.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc5fffff640 20 reqs_thread_entry: start 2024-03-04T17:53:12.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 init_complete bucket index max shards: 11 2024-03-04T17:53:12.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 Filter name: none 2024-03-04T17:53:12.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc5fdffb640 20 reqs_thread_entry: start 2024-03-04T17:53:12.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-maven : miss 2024-03-04T17:53:12.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x0 2024-03-04T17:53:12.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 adding default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-03-04T17:53:12.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 cache get: name=default.rgw.meta+users.email+sigv4@example.com : miss 2024-03-04T17:53:12.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x0 2024-03-04T17:53:12.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 adding default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-03-04T17:53:12.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : miss 2024-03-04T17:53:12.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 rados->read ofs=0 len=0 2024-03-04T17:53:12.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:53:12.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x0 2024-03-04T17:53:12.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 adding default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-03-04T17:53:12.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-03-04T17:53:12.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.411+0000 7fc7d11b5980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-03-04T17:53:12.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.415+0000 7fc7d11b5980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-03-04T17:53:12.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.415+0000 7fc7d11b5980 10 moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-03-04T17:53:12.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.415+0000 7fc7d11b5980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-maven, ofs0, ns] 2024-03-04T17:53:12.412 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.415+0000 7fc6b27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1065151889420 cookie 94809294471728 notifier 5444 bl.length()=644 2024-03-04T17:53:12.412 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.415+0000 7fc6b27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-03-04T17:53:12.412 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.415+0000 7fc6b27fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-03-04T17:53:12.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.423+0000 7fc7d11b5980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-03-04T17:53:12.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.423+0000 7fc7d11b5980 10 moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-03-04T17:53:12.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.423+0000 7fc7d11b5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:sigv4@example.com, ofs0, ns] 2024-03-04T17:53:12.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.423+0000 7fc6b27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1065151889418 cookie 94809294413616 notifier 5444 bl.length()=187 2024-03-04T17:53:12.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.423+0000 7fc6b27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-03-04T17:53:12.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.423+0000 7fc6b27fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-03-04T17:53:12.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.427+0000 7fc7d11b5980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-03-04T17:53:12.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.427+0000 7fc7d11b5980 10 moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-03-04T17:53:12.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.427+0000 7fc7d11b5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:lNCnR47C2g+ZidCWBAUuwfSAA7Q=, ofs0, ns] 2024-03-04T17:53:12.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.427+0000 7fc6b1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1065151889419 cookie 94809294413616 notifier 5444 bl.length()=197 2024-03-04T17:53:12.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.427+0000 7fc6b1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-03-04T17:53:12.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.427+0000 7fc6b1ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-03-04T17:53:12.423 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:53:12.423 INFO:tasks.workunit.client.0.smithi047.stdout: "user_id": "ceph-test-maven", 2024-03-04T17:53:12.423 INFO:tasks.workunit.client.0.smithi047.stdout: "display_name": "maven test user", 2024-03-04T17:53:12.423 INFO:tasks.workunit.client.0.smithi047.stdout: "email": "sigv4@example.com", 2024-03-04T17:53:12.423 INFO:tasks.workunit.client.0.smithi047.stdout: "suspended": 0, 2024-03-04T17:53:12.423 INFO:tasks.workunit.client.0.smithi047.stdout: "max_buckets": 1000, 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "subusers": [], 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "keys": [ 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: { 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "user": "ceph-test-maven", 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "access_key": "lNCnR47C2g+ZidCWBAUuwfSAA7Q=", 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "secret_key": "tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g==", 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "active": true 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: ], 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "swift_keys": [], 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "caps": [], 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "default_placement": "", 2024-03-04T17:53:12.424 INFO:tasks.workunit.client.0.smithi047.stdout: "default_storage_class": "", 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "placement_tags": [], 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_quota": { 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "enabled": false, 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size": -1, 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "max_objects": -1 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "user_quota": { 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "enabled": false, 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size": -1, 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: "max_objects": -1 2024-03-04T17:53:12.425 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:53:12.426 INFO:tasks.workunit.client.0.smithi047.stdout: "temp_url_keys": [], 2024-03-04T17:53:12.426 INFO:tasks.workunit.client.0.smithi047.stdout: "type": "rgw", 2024-03-04T17:53:12.426 INFO:tasks.workunit.client.0.smithi047.stdout: "mfa_ids": [] 2024-03-04T17:53:12.426 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:53:12.426 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:53:12.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.431+0000 7fc6857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:53:12.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.435+0000 7fc7d11b5980 20 remove_watcher() i=0 2024-03-04T17:53:12.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.435+0000 7fc7d11b5980 2 removed watcher, disabling cache 2024-03-04T17:53:12.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.435+0000 7fc7d11b5980 20 remove_watcher() i=1 2024-03-04T17:53:12.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.439+0000 7fc7d11b5980 20 remove_watcher() i=2 2024-03-04T17:53:12.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.439+0000 7fc7d11b5980 20 remove_watcher() i=3 2024-03-04T17:53:12.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.443+0000 7fc7d11b5980 20 remove_watcher() i=4 2024-03-04T17:53:12.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.447+0000 7fc7d11b5980 20 remove_watcher() i=5 2024-03-04T17:53:12.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.447+0000 7fc7d11b5980 20 remove_watcher() i=6 2024-03-04T17:53:12.446 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T17:53:12.451+0000 7fc7d11b5980 20 remove_watcher() i=7 2024-03-04T17:53:12.462 INFO:tasks.workunit.client.0.smithi047.stderr:+ [ -z ] 2024-03-04T17:53:12.462 INFO:tasks.workunit.client.0.smithi047.stderr:+ [ -z /home/ubuntu/cephtest ] 2024-03-04T17:53:12.463 INFO:tasks.workunit.client.0.smithi047.stderr:+ cat /home/ubuntu/cephtest/url_file 2024-03-04T17:53:12.464 INFO:tasks.workunit.client.0.smithi047.stderr:+ export RGW_HTTP_ENDPOINT_URL=https://smithi047.front.sepia.ceph.com:443 2024-03-04T17:53:12.464 INFO:tasks.workunit.client.0.smithi047.stderr:+ [ -z /home/ubuntu/cephtest/clone.client.0 ] 2024-03-04T17:53:12.464 INFO:tasks.workunit.client.0.smithi047.stderr:+ cd /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum 2024-03-04T17:53:12.464 INFO:tasks.workunit.client.0.smithi047.stderr:+ ./mvnw clean package 2024-03-04T17:53:12.471 INFO:tasks.workunit.client.0.smithi047.stdout:Warning: JAVA_HOME environment variable is not set. 2024-03-04T17:53:16.130 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Scanning for projects... 2024-03-04T17:53:16.275 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom 2024-03-04T17:53:16.471 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/5.6 kB Progress (1): 2.8/5.6 kB Progress (1): 4.1/5.6 kB Progress (1): 5.5/5.6 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom (5.6 kB at 28 kB/s) 2024-03-04T17:53:16.488 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom 2024-03-04T17:53:16.592 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/83 kB Progress (1): 33/83 kB Progress (1): 49/83 kB Progress (1): 66/83 kB Progress (1): 82/83 kB Progress (1): 83 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom (83 kB at 805 kB/s) 2024-03-04T17:53:16.618 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom 2024-03-04T17:53:16.680 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/77 kB Progress (1): 32/77 kB Progress (1): 49/77 kB Progress (1): 65/77 kB Progress (1): 77 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom (77 kB at 1.2 MB/s) 2024-03-04T17:53:16.722 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:16.722 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-03-04T17:53:16.722 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Building jcksum 1.0 2024-03-04T17:53:16.722 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] from pom.xml 2024-03-04T17:53:16.722 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-03-04T17:53:16.727 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom 2024-03-04T17:53:16.772 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom (5.3 kB at 118 kB/s) 2024-03-04T17:53:16.782 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom 2024-03-04T17:53:16.834 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom (9.9 kB at 190 kB/s) 2024-03-04T17:53:16.841 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom 2024-03-04T17:53:16.896 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom (45 kB at 823 kB/s) 2024-03-04T17:53:16.909 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom 2024-03-04T17:53:16.958 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom (21 kB at 419 kB/s) 2024-03-04T17:53:16.988 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar 2024-03-04T17:53:17.041 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/36 kB Progress (1): 32/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar (36 kB at 661 kB/s) 2024-03-04T17:53:17.060 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom 2024-03-04T17:53:17.109 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom (8.2 kB at 166 kB/s) 2024-03-04T17:53:17.116 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom 2024-03-04T17:53:17.164 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom (8.1 kB at 169 kB/s) 2024-03-04T17:53:17.170 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom 2024-03-04T17:53:17.226 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/48 kB Progress (1): 32/48 kB Progress (1): 48 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom (48 kB at 857 kB/s) 2024-03-04T17:53:17.237 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom 2024-03-04T17:53:17.291 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom (21 kB at 384 kB/s) 2024-03-04T17:53:17.305 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar 2024-03-04T17:53:17.367 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar (31 kB at 491 kB/s) 2024-03-04T17:53:17.381 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom 2024-03-04T17:53:17.442 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 9.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom (9.8 kB at 161 kB/s) 2024-03-04T17:53:17.458 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar 2024-03-04T17:53:17.516 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/66 kB Progress (1): 33/66 kB Progress (1): 49/66 kB Progress (1): 66/66 kB Progress (1): 66 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar (66 kB at 1.1 MB/s) 2024-03-04T17:53:17.532 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom 2024-03-04T17:53:17.631 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom (5.3 kB at 54 kB/s) 2024-03-04T17:53:17.638 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom 2024-03-04T17:53:17.684 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom (22 kB at 477 kB/s) 2024-03-04T17:53:17.695 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom 2024-03-04T17:53:17.762 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/50 kB Progress (1): 32/50 kB Progress (1): 49/50 kB Progress (1): 50 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom (50 kB at 744 kB/s) 2024-03-04T17:53:17.778 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom 2024-03-04T17:53:17.841 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom (24 kB at 374 kB/s) 2024-03-04T17:53:17.854 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom 2024-03-04T17:53:17.910 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom (5.6 kB at 101 kB/s) 2024-03-04T17:53:17.915 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar 2024-03-04T17:53:17.983 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/45 kB Progress (1): 32/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar (45 kB at 646 kB/s) 2024-03-04T17:53:18.002 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom 2024-03-04T17:53:18.055 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom (6.8 kB at 125 kB/s) 2024-03-04T17:53:18.062 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom 2024-03-04T17:53:18.121 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom (9.9 kB at 165 kB/s) 2024-03-04T17:53:18.127 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom 2024-03-04T17:53:18.197 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/46 kB Progress (1): 33/46 kB Progress (1): 46 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom (46 kB at 652 kB/s) 2024-03-04T17:53:18.205 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom 2024-03-04T17:53:18.261 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom (20 kB at 364 kB/s) 2024-03-04T17:53:18.273 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar 2024-03-04T17:53:18.338 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar (27 kB at 421 kB/s) 2024-03-04T17:53:18.376 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom 2024-03-04T17:53:18.438 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom (6.1 kB at 97 kB/s) 2024-03-04T17:53:18.443 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom 2024-03-04T17:53:18.508 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom (21 kB at 328 kB/s) 2024-03-04T17:53:18.520 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom 2024-03-04T17:53:18.594 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom (20 kB at 270 kB/s) 2024-03-04T17:53:18.612 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom 2024-03-04T17:53:18.676 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom (4.0 kB at 63 kB/s) 2024-03-04T17:53:18.681 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom 2024-03-04T17:53:18.739 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom (1.6 kB at 26 kB/s) 2024-03-04T17:53:18.744 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom 2024-03-04T17:53:18.802 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom (2.2 kB at 38 kB/s) 2024-03-04T17:53:18.813 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom 2024-03-04T17:53:18.872 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom (3.6 kB at 62 kB/s) 2024-03-04T17:53:18.882 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom 2024-03-04T17:53:18.941 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom (3.2 kB at 56 kB/s) 2024-03-04T17:53:18.951 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom 2024-03-04T17:53:19.011 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom (11 kB at 180 kB/s) 2024-03-04T17:53:19.022 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom 2024-03-04T17:53:19.082 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom (1.7 kB at 28 kB/s) 2024-03-04T17:53:19.092 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom 2024-03-04T17:53:19.150 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom (3.8 kB at 65 kB/s) 2024-03-04T17:53:19.159 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom 2024-03-04T17:53:19.215 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom (4.9 kB at 88 kB/s) 2024-03-04T17:53:19.224 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom 2024-03-04T17:53:19.283 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom (1.2 kB at 19 kB/s) 2024-03-04T17:53:19.288 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom 2024-03-04T17:53:19.346 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom (3.8 kB at 66 kB/s) 2024-03-04T17:53:19.350 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom 2024-03-04T17:53:19.412 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom (14 kB at 223 kB/s) 2024-03-04T17:53:19.421 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom 2024-03-04T17:53:19.478 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom (3.7 kB at 63 kB/s) 2024-03-04T17:53:19.492 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom 2024-03-04T17:53:19.552 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom (2.5 kB at 40 kB/s) 2024-03-04T17:53:19.562 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom 2024-03-04T17:53:19.621 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom (2.9 kB at 50 kB/s) 2024-03-04T17:53:19.635 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom 2024-03-04T17:53:19.695 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 7.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom (7.4 kB at 121 kB/s) 2024-03-04T17:53:19.706 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom 2024-03-04T17:53:19.767 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom (5.5 kB at 89 kB/s) 2024-03-04T17:53:19.777 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom 2024-03-04T17:53:19.819 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom (3.5 kB at 83 kB/s) 2024-03-04T17:53:19.829 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom 2024-03-04T17:53:19.893 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom (4.4 kB at 69 kB/s) 2024-03-04T17:53:19.898 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom 2024-03-04T17:53:19.945 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom (2.4 kB at 50 kB/s) 2024-03-04T17:53:19.954 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom 2024-03-04T17:53:20.006 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom (6.4 kB at 124 kB/s) 2024-03-04T17:53:20.012 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom 2024-03-04T17:53:20.067 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom (6.1 kB at 110 kB/s) 2024-03-04T17:53:20.077 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom 2024-03-04T17:53:20.124 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom (3.0 kB at 63 kB/s) 2024-03-04T17:53:20.133 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom 2024-03-04T17:53:20.187 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom (3.5 kB at 63 kB/s) 2024-03-04T17:53:20.197 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom 2024-03-04T17:53:20.252 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom (4.1 kB at 74 kB/s) 2024-03-04T17:53:20.257 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom 2024-03-04T17:53:20.312 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom (1.8 kB at 32 kB/s) 2024-03-04T17:53:20.323 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom 2024-03-04T17:53:20.373 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (6.6 kB at 130 kB/s) 2024-03-04T17:53:20.380 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom 2024-03-04T17:53:20.443 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/16 kB Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom (16 kB at 260 kB/s) 2024-03-04T17:53:20.452 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom 2024-03-04T17:53:20.489 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/35 kB Progress (1): 33/35 kB Progress (1): 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom (35 kB at 938 kB/s) 2024-03-04T17:53:20.497 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom 2024-03-04T17:53:20.527 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom (17 kB at 570 kB/s) 2024-03-04T17:53:20.536 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom 2024-03-04T17:53:20.563 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom (5.0 kB at 177 kB/s) 2024-03-04T17:53:20.568 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom 2024-03-04T17:53:20.595 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom (13 kB at 470 kB/s) 2024-03-04T17:53:20.605 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom 2024-03-04T17:53:20.633 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom (14 kB at 499 kB/s) 2024-03-04T17:53:20.638 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom 2024-03-04T17:53:20.674 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/68 kB Progress (1): 33/68 kB Progress (1): 49/68 kB Progress (1): 66/68 kB Progress (1): 68 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom (68 kB at 1.9 MB/s) 2024-03-04T17:53:20.702 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2024-03-04T17:53:20.732 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom (16 kB at 522 kB/s) 2024-03-04T17:53:20.741 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom 2024-03-04T17:53:20.771 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom (15 kB at 515 kB/s) 2024-03-04T17:53:20.776 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom 2024-03-04T17:53:20.818 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/79 kB Progress (1): 33/79 kB Progress (1): 49/79 kB Progress (1): 66/79 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom (79 kB at 1.9 MB/s) 2024-03-04T17:53:20.829 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom 2024-03-04T17:53:20.859 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom (18 kB at 614 kB/s) 2024-03-04T17:53:20.871 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom 2024-03-04T17:53:20.904 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom (20 kB at 624 kB/s) 2024-03-04T17:53:20.912 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom 2024-03-04T17:53:20.950 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 37/78 kB Progress (1): 53/78 kB Progress (1): 69/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom (78 kB at 2.0 MB/s) 2024-03-04T17:53:20.969 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom 2024-03-04T17:53:20.996 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom (3.8 kB at 142 kB/s) 2024-03-04T17:53:21.000 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom 2024-03-04T17:53:21.031 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom (14 kB at 445 kB/s) 2024-03-04T17:53:21.038 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom 2024-03-04T17:53:21.065 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom (3.2 kB at 119 kB/s) 2024-03-04T17:53:21.071 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom 2024-03-04T17:53:21.098 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom (3.2 kB at 118 kB/s) 2024-03-04T17:53:21.103 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom 2024-03-04T17:53:21.129 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom (2.0 kB at 78 kB/s) 2024-03-04T17:53:21.134 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom 2024-03-04T17:53:21.161 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom (2.8 kB at 105 kB/s) 2024-03-04T17:53:21.167 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom 2024-03-04T17:53:21.194 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom (1.5 kB at 54 kB/s) 2024-03-04T17:53:21.199 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom 2024-03-04T17:53:21.227 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom (3.0 kB at 108 kB/s) 2024-03-04T17:53:21.233 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom 2024-03-04T17:53:21.261 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom (3.2 kB at 110 kB/s) 2024-03-04T17:53:21.266 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom 2024-03-04T17:53:21.294 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom (3.2 kB at 111 kB/s) 2024-03-04T17:53:21.300 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom 2024-03-04T17:53:21.329 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom (1.0 kB at 35 kB/s) 2024-03-04T17:53:21.338 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom 2024-03-04T17:53:21.366 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 964 B Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom (964 B at 33 kB/s) 2024-03-04T17:53:21.373 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom 2024-03-04T17:53:21.401 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom (3.1 kB at 109 kB/s) 2024-03-04T17:53:21.407 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom 2024-03-04T17:53:21.434 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom (3.0 kB at 108 kB/s) 2024-03-04T17:53:21.439 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom 2024-03-04T17:53:21.468 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom (3.4 kB at 119 kB/s) 2024-03-04T17:53:21.474 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom 2024-03-04T17:53:21.504 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom (3.4 kB at 111 kB/s) 2024-03-04T17:53:21.509 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom 2024-03-04T17:53:21.538 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom (3.0 kB at 108 kB/s) 2024-03-04T17:53:21.606 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar 2024-03-04T17:53:22.043 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3 MB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar (3.3 MB at 7.5 MB/s) 2024-03-04T17:53:22.047 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar 2024-03-04T17:53:22.047 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar 2024-03-04T17:53:22.047 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar 2024-03-04T17:53:22.048 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar 2024-03-04T17:53:22.049 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar 2024-03-04T17:53:22.081 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/71 kB Progress (1): 32/71 kB Progress (1): 49/71 kB Progress (1): 65/71 kB Progress (1): 71 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar (71 kB at 2.0 MB/s) 2024-03-04T17:53:22.081 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar 2024-03-04T17:53:22.126 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/16 kB Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar (16 kB at 211 kB/s) 2024-03-04T17:53:22.126 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar 2024-03-04T17:53:22.135 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/16 kB Progress (1): 2.8/16 kB Progress (1): 4.1/16 kB Progress (1): 5.5/16 kB Progress (1): 6.9/16 kB Progress (1): 8.3/16 kB Progress (1): 9.7/16 kB Progress (1): 11/16 kB Progress (1): 12/16 kB Progress (1): 14/16 kB Progress (1): 15/16 kB Progress (1): 16 kB Progress (2): 16 kB | 1.4/49 kB Progress (2): 16 kB | 2.8/49 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar (16 kB at 182 kB/s) 2024-03-04T17:53:22.135 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.1/49 kB Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar 2024-03-04T17:53:22.164 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.5/49 kB Progress (1): 6.9/49 kB Progress (1): 8.3/49 kB Progress (1): 9.7/49 kB Progress (1): 11/49 kB Progress (1): 12/49 kB Progress (1): 14/49 kB Progress (1): 15/49 kB Progress (1): 17/49 kB Progress (1): 18/49 kB Progress (1): 19/49 kB Progress (1): 21/49 kB Progress (1): 22/49 kB Progress (1): 23/49 kB Progress (1): 25/49 kB Progress (1): 26/49 kB Progress (1): 28/49 kB Progress (1): 29/49 kB Progress (1): 30/49 kB Progress (1): 32/49 kB Progress (1): 33/49 kB Progress (1): 34/49 kB Progress (1): 36/49 kB Progress (1): 37/49 kB Progress (1): 39/49 kB Progress (2): 39/49 kB | 16/782 kB Progress (2): 39/49 kB | 33/782 kB Progress (2): 39/49 kB | 49/782 kB Progress (2): 39/49 kB | 66/782 kB Progress (2): 39/49 kB | 82/782 kB Progress (2): 40/49 kB | 82/782 kB Progress (2): 41/49 kB | 82/782 kB Progress (2): 43/49 kB | 82/782 kB Progress (2): 44/49 kB | 82/782 kB Progress (2): 46/49 kB | 82/782 kB Progress (2): 47/49 kB | 82/782 kB Progress (2): 47/49 kB | 98/782 kB Progress (3): 47/49 kB | 98/782 kB | 16/98 kB Progress (3): 48/49 kB | 98/782 kB | 16/98 kB Progress (4): 48/49 kB | 98/782 kB | 16/98 kB | 1.4/11 kB Progress (4): 49 kB | 98/782 kB | 16/98 kB | 1.4/11 kB Progress (4): 49 kB | 115/782 kB | 16/98 kB | 1.4/11 kB Progress (4): 49 kB | 115/782 kB | 16/98 kB | 2.8/11 kB Progress (4): 49 kB | 115/782 kB | 16/98 kB | 4.1/11 kB Progress (4): 49 kB | 131/782 kB | 16/98 kB | 4.1/11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar (49 kB at 428 kB/s) 2024-03-04T17:53:22.164 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 131/782 kB | 16/98 kB | 5.5/11 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar 2024-03-04T17:53:22.167 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 131/782 kB | 16/98 kB | 6.9/11 kB Progress (3): 147/782 kB | 16/98 kB | 6.9/11 kB Progress (3): 147/782 kB | 16/98 kB | 8.3/11 kB Progress (3): 147/782 kB | 16/98 kB | 9.7/11 kB Progress (4): 147/782 kB | 16/98 kB | 9.7/11 kB | 1.4/35 kB Progress (4): 147/782 kB | 16/98 kB | 9.7/11 kB | 2.8/35 kB Progress (4): 147/782 kB | 16/98 kB | 11/11 kB | 2.8/35 kB Progress (4): 147/782 kB | 16/98 kB | 11/11 kB | 4.1/35 kB Progress (4): 147/782 kB | 16/98 kB | 11 kB | 4.1/35 kB Progress (4): 164/782 kB | 16/98 kB | 11 kB | 4.1/35 kB Progress (4): 164/782 kB | 16/98 kB | 11 kB | 5.5/35 kB Progress (4): 164/782 kB | 16/98 kB | 11 kB | 6.9/35 kB Progress (4): 180/782 kB | 16/98 kB | 11 kB | 6.9/35 kB Progress (4): 180/782 kB | 33/98 kB | 11 kB | 6.9/35 kB Progress (4): 180/782 kB | 33/98 kB | 11 kB | 8.3/35 kB Progress (4): 197/782 kB | 33/98 kB | 11 kB | 8.3/35 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar (11 kB at 97 kB/s) 2024-03-04T17:53:22.167 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar 2024-03-04T17:53:22.175 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 197/782 kB | 33/98 kB | 9.7/35 kB Progress (3): 197/782 kB | 33/98 kB | 11/35 kB Progress (3): 213/782 kB | 33/98 kB | 11/35 kB Progress (3): 213/782 kB | 33/98 kB | 12/35 kB Progress (3): 213/782 kB | 33/98 kB | 14/35 kB Progress (3): 213/782 kB | 33/98 kB | 15/35 kB Progress (3): 213/782 kB | 33/98 kB | 17/35 kB Progress (3): 213/782 kB | 33/98 kB | 18/35 kB Progress (3): 213/782 kB | 33/98 kB | 19/35 kB Progress (3): 229/782 kB | 33/98 kB | 19/35 kB Progress (3): 229/782 kB | 33/98 kB | 21/35 kB Progress (3): 229/782 kB | 33/98 kB | 22/35 kB Progress (3): 246/782 kB | 33/98 kB | 22/35 kB Progress (3): 246/782 kB | 33/98 kB | 23/35 kB Progress (3): 246/782 kB | 33/98 kB | 25/35 kB Progress (3): 256/782 kB | 33/98 kB | 25/35 kB Progress (3): 256/782 kB | 33/98 kB | 26/35 kB Progress (3): 256/782 kB | 33/98 kB | 28/35 kB Progress (3): 256/782 kB | 33/98 kB | 29/35 kB Progress (3): 256/782 kB | 33/98 kB | 30/35 kB Progress (3): 272/782 kB | 33/98 kB | 30/35 kB Progress (3): 272/782 kB | 33/98 kB | 32/35 kB Progress (3): 272/782 kB | 33/98 kB | 33/35 kB Progress (3): 289/782 kB | 33/98 kB | 33/35 kB Progress (3): 289/782 kB | 33/98 kB | 34/35 kB Progress (3): 289/782 kB | 33/98 kB | 35 kB Progress (3): 305/782 kB | 33/98 kB | 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar (35 kB at 275 kB/s) 2024-03-04T17:53:22.175 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar 2024-03-04T17:53:22.198 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 321/782 kB | 33/98 kB Progress (2): 338/782 kB | 33/98 kB Progress (2): 354/782 kB | 33/98 kB Progress (2): 370/782 kB | 33/98 kB Progress (2): 387/782 kB | 33/98 kB Progress (2): 403/782 kB | 33/98 kB Progress (2): 420/782 kB | 33/98 kB Progress (2): 420/782 kB | 49/98 kB Progress (2): 436/782 kB | 49/98 kB Progress (2): 452/782 kB | 49/98 kB Progress (2): 469/782 kB | 49/98 kB Progress (2): 485/782 kB | 49/98 kB Progress (2): 485/782 kB | 66/98 kB Progress (2): 502/782 kB | 66/98 kB Progress (2): 518/782 kB | 66/98 kB Progress (3): 518/782 kB | 66/98 kB | 16/205 kB Progress (3): 518/782 kB | 82/98 kB | 16/205 kB Progress (3): 534/782 kB | 82/98 kB | 16/205 kB Progress (4): 534/782 kB | 82/98 kB | 16/205 kB | 1.4/30 kB Progress (4): 534/782 kB | 82/98 kB | 16/205 kB | 2.8/30 kB Progress (4): 534/782 kB | 82/98 kB | 16/205 kB | 4.1/30 kB Progress (4): 551/782 kB | 82/98 kB | 16/205 kB | 4.1/30 kB Progress (4): 551/782 kB | 82/98 kB | 16/205 kB | 5.5/30 kB Progress (4): 551/782 kB | 82/98 kB | 16/205 kB | 6.9/30 kB Progress (4): 551/782 kB | 82/98 kB | 32/205 kB | 6.9/30 kB Progress (4): 567/782 kB | 82/98 kB | 32/205 kB | 6.9/30 kB Progress (4): 567/782 kB | 82/98 kB | 32/205 kB | 8.3/30 kB Progress (4): 567/782 kB | 98 kB | 32/205 kB | 8.3/30 kB Progress (4): 567/782 kB | 98 kB | 32/205 kB | 9.7/30 kB Progress (4): 567/782 kB | 98 kB | 32/205 kB | 11/30 kB Progress (4): 583/782 kB | 98 kB | 32/205 kB | 11/30 kB Progress (4): 583/782 kB | 98 kB | 32/205 kB | 12/30 kB Progress (4): 583/782 kB | 98 kB | 49/205 kB | 12/30 kB Progress (4): 583/782 kB | 98 kB | 49/205 kB | 14/30 kB Progress (4): 600/782 kB | 98 kB | 49/205 kB | 14/30 kB Progress (4): 600/782 kB | 98 kB | 49/205 kB | 15/30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar (98 kB at 633 kB/s) 2024-03-04T17:53:22.198 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar 2024-03-04T17:53:22.205 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 600/782 kB | 49/205 kB | 17/30 kB Progress (3): 600/782 kB | 49/205 kB | 18/30 kB Progress (3): 600/782 kB | 49/205 kB | 19/30 kB Progress (3): 616/782 kB | 49/205 kB | 19/30 kB Progress (3): 616/782 kB | 49/205 kB | 21/30 kB Progress (3): 616/782 kB | 49/205 kB | 22/30 kB Progress (3): 616/782 kB | 49/205 kB | 23/30 kB Progress (3): 616/782 kB | 65/205 kB | 23/30 kB Progress (3): 616/782 kB | 65/205 kB | 25/30 kB Progress (3): 616/782 kB | 65/205 kB | 26/30 kB Progress (3): 633/782 kB | 65/205 kB | 26/30 kB Progress (3): 633/782 kB | 65/205 kB | 28/30 kB Progress (3): 633/782 kB | 65/205 kB | 29/30 kB Progress (3): 633/782 kB | 65/205 kB | 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar (30 kB at 195 kB/s) 2024-03-04T17:53:22.205 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar 2024-03-04T17:53:22.218 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 633/782 kB | 65/205 kB | 1.4/81 kB Progress (3): 633/782 kB | 65/205 kB | 2.8/81 kB Progress (3): 633/782 kB | 65/205 kB | 4.1/81 kB Progress (3): 633/782 kB | 65/205 kB | 5.5/81 kB Progress (3): 649/782 kB | 65/205 kB | 5.5/81 kB Progress (3): 649/782 kB | 65/205 kB | 6.9/81 kB Progress (3): 649/782 kB | 65/205 kB | 8.3/81 kB Progress (3): 649/782 kB | 65/205 kB | 9.7/81 kB Progress (3): 665/782 kB | 65/205 kB | 9.7/81 kB Progress (3): 665/782 kB | 65/205 kB | 11/81 kB Progress (3): 665/782 kB | 65/205 kB | 12/81 kB Progress (3): 682/782 kB | 65/205 kB | 12/81 kB Progress (3): 682/782 kB | 65/205 kB | 14/81 kB Progress (3): 682/782 kB | 65/205 kB | 15/81 kB Progress (3): 682/782 kB | 65/205 kB | 16/81 kB Progress (3): 698/782 kB | 65/205 kB | 16/81 kB Progress (3): 715/782 kB | 65/205 kB | 16/81 kB Progress (3): 715/782 kB | 65/205 kB | 32/81 kB Progress (3): 731/782 kB | 65/205 kB | 32/81 kB Progress (3): 747/782 kB | 65/205 kB | 32/81 kB Progress (3): 747/782 kB | 65/205 kB | 49/81 kB Progress (3): 764/782 kB | 65/205 kB | 49/81 kB Progress (3): 780/782 kB | 65/205 kB | 49/81 kB Progress (3): 782 kB | 65/205 kB | 49/81 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar (782 kB at 4.6 MB/s) 2024-03-04T17:53:22.218 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 81/205 kB | 49/81 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar 2024-03-04T17:53:22.231 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 98/205 kB | 49/81 kB Progress (2): 114/205 kB | 49/81 kB Progress (2): 131/205 kB | 49/81 kB Progress (2): 147/205 kB | 49/81 kB Progress (2): 163/205 kB | 49/81 kB Progress (2): 180/205 kB | 49/81 kB Progress (2): 196/205 kB | 49/81 kB Progress (3): 196/205 kB | 49/81 kB | 16/798 kB Progress (3): 205 kB | 49/81 kB | 16/798 kB Progress (3): 205 kB | 49/81 kB | 33/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar (205 kB at 1.1 MB/s) 2024-03-04T17:53:22.232 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar 2024-03-04T17:53:22.236 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 65/81 kB | 33/798 kB Progress (2): 65/81 kB | 49/798 kB Progress (2): 65/81 kB | 66/798 kB Progress (3): 65/81 kB | 66/798 kB | 13 kB Progress (3): 65/81 kB | 82/798 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar (13 kB at 72 kB/s) 2024-03-04T17:53:22.236 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar 2024-03-04T17:53:22.238 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 65/81 kB | 98/798 kB Progress (2): 81 kB | 98/798 kB Progress (2): 81 kB | 115/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar (81 kB at 432 kB/s) 2024-03-04T17:53:22.238 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar 2024-03-04T17:53:22.265 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 131/798 kB Progress (2): 131/798 kB | 16/182 kB Progress (2): 131/798 kB | 33/182 kB Progress (2): 131/798 kB | 49/182 kB Progress (2): 131/798 kB | 62/182 kB Progress (2): 131/798 kB | 79/182 kB Progress (2): 131/798 kB | 95/182 kB Progress (2): 131/798 kB | 111/182 kB Progress (2): 147/798 kB | 111/182 kB Progress (2): 147/798 kB | 128/182 kB Progress (2): 164/798 kB | 128/182 kB Progress (2): 164/798 kB | 144/182 kB Progress (2): 180/798 kB | 144/182 kB Progress (2): 180/798 kB | 160/182 kB Progress (2): 197/798 kB | 160/182 kB Progress (2): 197/798 kB | 177/182 kB Progress (2): 197/798 kB | 182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar (182 kB at 855 kB/s) 2024-03-04T17:53:22.265 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar 2024-03-04T17:53:22.284 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 213/798 kB Progress (1): 229/798 kB Progress (2): 229/798 kB | 16/151 kB Progress (2): 246/798 kB | 16/151 kB Progress (2): 246/798 kB | 33/151 kB Progress (2): 256/798 kB | 33/151 kB Progress (2): 256/798 kB | 49/151 kB Progress (2): 272/798 kB | 49/151 kB Progress (2): 272/798 kB | 66/151 kB Progress (2): 272/798 kB | 82/151 kB Progress (2): 289/798 kB | 82/151 kB Progress (2): 289/798 kB | 98/151 kB Progress (2): 305/798 kB | 98/151 kB Progress (2): 321/798 kB | 98/151 kB Progress (2): 321/798 kB | 115/151 kB Progress (3): 321/798 kB | 115/151 kB | 16/27 kB Progress (3): 338/798 kB | 115/151 kB | 16/27 kB Progress (3): 338/798 kB | 131/151 kB | 16/27 kB Progress (3): 354/798 kB | 131/151 kB | 16/27 kB Progress (3): 354/798 kB | 147/151 kB | 16/27 kB Progress (3): 354/798 kB | 151 kB | 16/27 kB Progress (3): 354/798 kB | 151 kB | 27 kB Progress (4): 354/798 kB | 151 kB | 27 kB | 16/30 kB Progress (4): 370/798 kB | 151 kB | 27 kB | 16/30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar (151 kB at 650 kB/s) 2024-03-04T17:53:22.284 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 387/798 kB | 27 kB | 16/30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar (27 kB at 117 kB/s) 2024-03-04T17:53:22.284 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar 2024-03-04T17:53:22.284 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar 2024-03-04T17:53:22.286 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 403/798 kB | 16/30 kB Progress (2): 403/798 kB | 30 kB Progress (2): 420/798 kB | 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar (30 kB at 128 kB/s) 2024-03-04T17:53:22.286 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar 2024-03-04T17:53:22.322 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 436/798 kB Progress (1): 452/798 kB Progress (1): 469/798 kB Progress (1): 485/798 kB Progress (1): 502/798 kB Progress (1): 518/798 kB Progress (1): 534/798 kB Progress (1): 551/798 kB Progress (2): 551/798 kB | 16/385 kB Progress (2): 551/798 kB | 32/385 kB Progress (2): 567/798 kB | 32/385 kB Progress (2): 567/798 kB | 49/385 kB Progress (2): 583/798 kB | 49/385 kB Progress (2): 583/798 kB | 65/385 kB Progress (2): 583/798 kB | 81/385 kB Progress (2): 600/798 kB | 81/385 kB Progress (2): 600/798 kB | 98/385 kB Progress (2): 600/798 kB | 114/385 kB Progress (2): 616/798 kB | 114/385 kB Progress (2): 616/798 kB | 131/385 kB Progress (2): 616/798 kB | 147/385 kB Progress (2): 616/798 kB | 163/385 kB Progress (2): 616/798 kB | 180/385 kB Progress (2): 633/798 kB | 180/385 kB Progress (2): 633/798 kB | 196/385 kB Progress (2): 649/798 kB | 196/385 kB Progress (2): 649/798 kB | 212/385 kB Progress (2): 649/798 kB | 229/385 kB Progress (2): 665/798 kB | 229/385 kB Progress (2): 665/798 kB | 245/385 kB Progress (2): 682/798 kB | 245/385 kB Progress (2): 682/798 kB | 262/385 kB Progress (2): 698/798 kB | 262/385 kB Progress (2): 698/798 kB | 278/385 kB Progress (2): 698/798 kB | 294/385 kB Progress (3): 698/798 kB | 294/385 kB | 13 kB Progress (3): 715/798 kB | 294/385 kB | 13 kB Progress (3): 715/798 kB | 311/385 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar (13 kB at 49 kB/s) 2024-03-04T17:53:22.322 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar 2024-03-04T17:53:22.333 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 731/798 kB | 311/385 kB Progress (2): 731/798 kB | 327/385 kB Progress (3): 731/798 kB | 327/385 kB | 16/75 kB Progress (3): 731/798 kB | 344/385 kB | 16/75 kB Progress (4): 731/798 kB | 344/385 kB | 16/75 kB | 16/780 kB Progress (4): 731/798 kB | 344/385 kB | 32/75 kB | 16/780 kB Progress (4): 747/798 kB | 344/385 kB | 32/75 kB | 16/780 kB Progress (4): 747/798 kB | 344/385 kB | 32/75 kB | 33/780 kB Progress (4): 764/798 kB | 344/385 kB | 32/75 kB | 33/780 kB Progress (4): 780/798 kB | 344/385 kB | 32/75 kB | 33/780 kB Progress (4): 780/798 kB | 344/385 kB | 49/75 kB | 33/780 kB Progress (4): 780/798 kB | 360/385 kB | 49/75 kB | 33/780 kB Progress (4): 796/798 kB | 360/385 kB | 49/75 kB | 33/780 kB Progress (4): 798 kB | 360/385 kB | 49/75 kB | 33/780 kB Progress (4): 798 kB | 360/385 kB | 49/75 kB | 49/780 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar (798 kB at 2.8 MB/s) 2024-03-04T17:53:22.333 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar 2024-03-04T17:53:22.335 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 376/385 kB | 49/75 kB | 49/780 kB Progress (3): 385 kB | 49/75 kB | 49/780 kB Progress (3): 385 kB | 65/75 kB | 49/780 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar (385 kB at 1.4 MB/s) 2024-03-04T17:53:22.335 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar 2024-03-04T17:53:22.353 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 65/75 kB | 66/780 kB Progress (2): 65/75 kB | 82/780 kB Progress (2): 65/75 kB | 98/780 kB Progress (2): 65/75 kB | 115/780 kB Progress (2): 75 kB | 115/780 kB Progress (2): 75 kB | 131/780 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar (75 kB at 247 kB/s) 2024-03-04T17:53:22.353 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar 2024-03-04T17:53:22.397 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 147/780 kB Progress (2): 147/780 kB | 16/329 kB Progress (2): 147/780 kB | 33/329 kB Progress (2): 147/780 kB | 37/329 kB Progress (2): 164/780 kB | 37/329 kB Progress (2): 164/780 kB | 53/329 kB Progress (2): 164/780 kB | 69/329 kB Progress (2): 180/780 kB | 69/329 kB Progress (2): 180/780 kB | 86/329 kB Progress (2): 180/780 kB | 102/329 kB Progress (2): 197/780 kB | 102/329 kB Progress (2): 197/780 kB | 118/329 kB Progress (3): 197/780 kB | 118/329 kB | 16/501 kB Progress (3): 197/780 kB | 135/329 kB | 16/501 kB Progress (4): 197/780 kB | 135/329 kB | 16/501 kB | 16/354 kB Progress (4): 197/780 kB | 135/329 kB | 33/501 kB | 16/354 kB Progress (4): 213/780 kB | 135/329 kB | 33/501 kB | 16/354 kB Progress (4): 213/780 kB | 135/329 kB | 33/501 kB | 33/354 kB Progress (4): 213/780 kB | 151/329 kB | 33/501 kB | 33/354 kB Progress (4): 213/780 kB | 151/329 kB | 49/501 kB | 33/354 kB Progress (4): 213/780 kB | 151/329 kB | 49/501 kB | 49/354 kB Progress (4): 213/780 kB | 151/329 kB | 66/501 kB | 49/354 kB Progress (4): 213/780 kB | 168/329 kB | 66/501 kB | 49/354 kB Progress (4): 213/780 kB | 168/329 kB | 82/501 kB | 49/354 kB Progress (4): 229/780 kB | 168/329 kB | 82/501 kB | 49/354 kB Progress (4): 229/780 kB | 168/329 kB | 82/501 kB | 66/354 kB Progress (4): 229/780 kB | 184/329 kB | 82/501 kB | 66/354 kB Progress (4): 229/780 kB | 184/329 kB | 98/501 kB | 66/354 kB Progress (4): 229/780 kB | 184/329 kB | 98/501 kB | 82/354 kB Progress (4): 229/780 kB | 184/329 kB | 115/501 kB | 82/354 kB Progress (4): 229/780 kB | 184/329 kB | 131/501 kB | 82/354 kB Progress (4): 229/780 kB | 184/329 kB | 131/501 kB | 98/354 kB Progress (4): 229/780 kB | 200/329 kB | 131/501 kB | 98/354 kB Progress (4): 229/780 kB | 200/329 kB | 147/501 kB | 98/354 kB Progress (4): 229/780 kB | 200/329 kB | 147/501 kB | 115/354 kB Progress (4): 246/780 kB | 200/329 kB | 147/501 kB | 115/354 kB Progress (4): 246/780 kB | 217/329 kB | 147/501 kB | 115/354 kB Progress (4): 246/780 kB | 233/329 kB | 147/501 kB | 115/354 kB Progress (4): 246/780 kB | 233/329 kB | 164/501 kB | 115/354 kB Progress (4): 246/780 kB | 233/329 kB | 164/501 kB | 131/354 kB Progress (4): 246/780 kB | 233/329 kB | 164/501 kB | 147/354 kB Progress (4): 256/780 kB | 233/329 kB | 164/501 kB | 147/354 kB Progress (4): 256/780 kB | 233/329 kB | 180/501 kB | 147/354 kB Progress (4): 256/780 kB | 250/329 kB | 180/501 kB | 147/354 kB Progress (4): 256/780 kB | 250/329 kB | 180/501 kB | 164/354 kB Progress (4): 256/780 kB | 250/329 kB | 197/501 kB | 164/354 kB Progress (4): 256/780 kB | 250/329 kB | 213/501 kB | 164/354 kB Progress (4): 256/780 kB | 266/329 kB | 213/501 kB | 164/354 kB Progress (4): 256/780 kB | 266/329 kB | 213/501 kB | 180/354 kB Progress (4): 256/780 kB | 282/329 kB | 213/501 kB | 180/354 kB Progress (4): 256/780 kB | 282/329 kB | 229/501 kB | 180/354 kB Progress (4): 256/780 kB | 282/329 kB | 246/501 kB | 180/354 kB Progress (4): 256/780 kB | 282/329 kB | 246/501 kB | 197/354 kB Progress (4): 256/780 kB | 282/329 kB | 256/501 kB | 197/354 kB Progress (4): 256/780 kB | 299/329 kB | 256/501 kB | 197/354 kB Progress (4): 272/780 kB | 299/329 kB | 256/501 kB | 197/354 kB Progress (4): 272/780 kB | 299/329 kB | 272/501 kB | 197/354 kB Progress (4): 272/780 kB | 299/329 kB | 272/501 kB | 213/354 kB Progress (4): 272/780 kB | 299/329 kB | 289/501 kB | 213/354 kB Progress (4): 272/780 kB | 299/329 kB | 289/501 kB | 229/354 kB Progress (4): 272/780 kB | 299/329 kB | 305/501 kB | 229/354 kB Progress (4): 272/780 kB | 299/329 kB | 305/501 kB | 246/354 kB Progress (4): 289/780 kB | 299/329 kB | 305/501 kB | 246/354 kB Progress (4): 289/780 kB | 299/329 kB | 305/501 kB | 256/354 kB Progress (4): 289/780 kB | 299/329 kB | 321/501 kB | 256/354 kB Progress (4): 289/780 kB | 315/329 kB | 321/501 kB | 256/354 kB Progress (4): 289/780 kB | 315/329 kB | 321/501 kB | 272/354 kB Progress (4): 289/780 kB | 315/329 kB | 338/501 kB | 272/354 kB Progress (4): 289/780 kB | 315/329 kB | 338/501 kB | 289/354 kB Progress (4): 289/780 kB | 329 kB | 338/501 kB | 289/354 kB Progress (4): 289/780 kB | 329 kB | 354/501 kB | 289/354 kB Progress (4): 289/780 kB | 329 kB | 354/501 kB | 305/354 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar (329 kB at 952 kB/s) 2024-03-04T17:53:22.397 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar 2024-03-04T17:53:22.410 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 305/780 kB | 354/501 kB | 305/354 kB Progress (3): 305/780 kB | 370/501 kB | 305/354 kB Progress (3): 305/780 kB | 370/501 kB | 321/354 kB Progress (4): 305/780 kB | 370/501 kB | 321/354 kB | 16/41 kB Progress (4): 321/780 kB | 370/501 kB | 321/354 kB | 16/41 kB Progress (4): 338/780 kB | 370/501 kB | 321/354 kB | 16/41 kB Progress (4): 338/780 kB | 370/501 kB | 321/354 kB | 32/41 kB Progress (4): 338/780 kB | 387/501 kB | 321/354 kB | 32/41 kB Progress (4): 338/780 kB | 387/501 kB | 338/354 kB | 32/41 kB Progress (4): 338/780 kB | 403/501 kB | 338/354 kB | 32/41 kB Progress (4): 338/780 kB | 403/501 kB | 354 kB | 32/41 kB Progress (4): 354/780 kB | 403/501 kB | 354 kB | 32/41 kB Progress (4): 354/780 kB | 403/501 kB | 354 kB | 41 kB Progress (4): 354/780 kB | 420/501 kB | 354 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar (354 kB at 988 kB/s) 2024-03-04T17:53:22.410 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar 2024-03-04T17:53:22.411 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 354/780 kB | 436/501 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar (41 kB at 115 kB/s) 2024-03-04T17:53:22.411 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar 2024-03-04T17:53:22.415 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 370/780 kB | 436/501 kB Progress (2): 370/780 kB | 452/501 kB Progress (2): 370/780 kB | 469/501 kB Progress (2): 370/780 kB | 485/501 kB Progress (2): 370/780 kB | 501 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar (501 kB at 1.4 MB/s) 2024-03-04T17:53:22.415 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar 2024-03-04T17:53:22.432 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 387/780 kB Progress (1): 403/780 kB Progress (1): 420/780 kB Progress (1): 436/780 kB Progress (1): 452/780 kB Progress (1): 469/780 kB Progress (2): 469/780 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar (6.4 kB at 17 kB/s) 2024-03-04T17:53:22.432 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar 2024-03-04T17:53:22.447 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 485/780 kB Progress (1): 502/780 kB Progress (1): 518/780 kB Progress (2): 518/780 kB | 16/211 kB Progress (2): 534/780 kB | 16/211 kB Progress (2): 534/780 kB | 33/211 kB Progress (2): 534/780 kB | 49/211 kB Progress (3): 534/780 kB | 49/211 kB | 16/586 kB Progress (4): 534/780 kB | 49/211 kB | 16/586 kB | 14 kB Progress (4): 534/780 kB | 66/211 kB | 16/586 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar (14 kB at 36 kB/s) 2024-03-04T17:53:22.447 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar 2024-03-04T17:53:22.461 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 551/780 kB | 66/211 kB | 16/586 kB Progress (3): 551/780 kB | 82/211 kB | 16/586 kB Progress (3): 551/780 kB | 98/211 kB | 16/586 kB Progress (3): 567/780 kB | 98/211 kB | 16/586 kB Progress (3): 567/780 kB | 115/211 kB | 16/586 kB Progress (3): 567/780 kB | 115/211 kB | 33/586 kB Progress (3): 567/780 kB | 131/211 kB | 33/586 kB Progress (3): 583/780 kB | 131/211 kB | 33/586 kB Progress (3): 583/780 kB | 147/211 kB | 33/586 kB Progress (3): 583/780 kB | 164/211 kB | 33/586 kB Progress (3): 600/780 kB | 164/211 kB | 33/586 kB Progress (3): 600/780 kB | 164/211 kB | 49/586 kB Progress (3): 600/780 kB | 180/211 kB | 49/586 kB Progress (3): 600/780 kB | 197/211 kB | 49/586 kB Progress (3): 616/780 kB | 197/211 kB | 49/586 kB Progress (3): 616/780 kB | 211 kB | 49/586 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar (211 kB at 517 kB/s) 2024-03-04T17:53:22.461 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar 2024-03-04T17:53:22.474 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 633/780 kB | 49/586 kB Progress (2): 633/780 kB | 66/586 kB Progress (2): 649/780 kB | 66/586 kB Progress (3): 649/780 kB | 66/586 kB | 16/106 kB Progress (3): 649/780 kB | 66/586 kB | 33/106 kB Progress (3): 665/780 kB | 66/586 kB | 33/106 kB Progress (3): 665/780 kB | 66/586 kB | 49/106 kB Progress (3): 665/780 kB | 66/586 kB | 66/106 kB Progress (3): 665/780 kB | 82/586 kB | 66/106 kB Progress (3): 682/780 kB | 82/586 kB | 66/106 kB Progress (3): 682/780 kB | 82/586 kB | 82/106 kB Progress (3): 682/780 kB | 82/586 kB | 98/106 kB Progress (3): 682/780 kB | 82/586 kB | 106 kB Progress (3): 698/780 kB | 82/586 kB | 106 kB Progress (3): 698/780 kB | 98/586 kB | 106 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar (106 kB at 252 kB/s) 2024-03-04T17:53:22.474 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar 2024-03-04T17:53:22.477 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 715/780 kB | 98/586 kB Progress (2): 715/780 kB | 115/586 kB Progress (3): 715/780 kB | 115/586 kB | 6.8 kB Progress (3): 731/780 kB | 115/586 kB | 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar (6.8 kB at 16 kB/s) 2024-03-04T17:53:22.477 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar 2024-03-04T17:53:22.486 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 747/780 kB | 115/586 kB Progress (2): 747/780 kB | 131/586 kB Progress (2): 764/780 kB | 131/586 kB Progress (2): 764/780 kB | 147/586 kB Progress (2): 780/780 kB | 147/586 kB Progress (2): 780 kB | 147/586 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar (780 kB at 1.8 MB/s) 2024-03-04T17:53:22.486 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar 2024-03-04T17:53:22.508 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 164/586 kB Progress (2): 164/586 kB | 16/245 kB Progress (2): 164/586 kB | 32/245 kB Progress (2): 164/586 kB | 49/245 kB Progress (2): 164/586 kB | 65/245 kB Progress (2): 180/586 kB | 65/245 kB Progress (2): 180/586 kB | 81/245 kB Progress (2): 180/586 kB | 98/245 kB Progress (2): 180/586 kB | 114/245 kB Progress (2): 180/586 kB | 131/245 kB Progress (2): 197/586 kB | 131/245 kB Progress (2): 197/586 kB | 147/245 kB Progress (2): 197/586 kB | 163/245 kB Progress (2): 197/586 kB | 180/245 kB Progress (2): 213/586 kB | 180/245 kB Progress (2): 213/586 kB | 196/245 kB Progress (2): 213/586 kB | 212/245 kB Progress (3): 213/586 kB | 212/245 kB | 16/205 kB Progress (3): 213/586 kB | 229/245 kB | 16/205 kB Progress (3): 213/586 kB | 229/245 kB | 33/205 kB Progress (3): 213/586 kB | 229/245 kB | 49/205 kB Progress (3): 213/586 kB | 245 kB | 49/205 kB Progress (3): 229/586 kB | 245 kB | 49/205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar (245 kB at 538 kB/s) 2024-03-04T17:53:22.508 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar 2024-03-04T17:53:22.510 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 229/586 kB | 66/205 kB Progress (3): 229/586 kB | 66/205 kB | 15 kB Progress (3): 229/586 kB | 82/205 kB | 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar (15 kB at 33 kB/s) 2024-03-04T17:53:22.510 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar 2024-03-04T17:53:22.520 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 229/586 kB | 98/205 kB Progress (2): 246/586 kB | 98/205 kB Progress (2): 246/586 kB | 115/205 kB Progress (2): 246/586 kB | 131/205 kB Progress (2): 256/586 kB | 131/205 kB Progress (2): 256/586 kB | 147/205 kB Progress (2): 256/586 kB | 164/205 kB Progress (2): 256/586 kB | 180/205 kB Progress (2): 256/586 kB | 197/205 kB Progress (2): 256/586 kB | 205 kB Progress (3): 256/586 kB | 205 kB | 16/16 kB Progress (3): 256/586 kB | 205 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar (16 kB at 35 kB/s) 2024-03-04T17:53:22.520 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 272/586 kB | 205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar (205 kB at 439 kB/s) 2024-03-04T17:53:22.520 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar 2024-03-04T17:53:22.520 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar 2024-03-04T17:53:22.539 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 289/586 kB Progress (1): 305/586 kB Progress (1): 321/586 kB Progress (1): 338/586 kB Progress (2): 338/586 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar (6.4 kB at 13 kB/s) 2024-03-04T17:53:22.539 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar 2024-03-04T17:53:22.541 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 354/586 kB Progress (2): 354/586 kB | 16/23 kB Progress (2): 354/586 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar (23 kB at 46 kB/s) 2024-03-04T17:53:22.552 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 370/586 kB Progress (1): 387/586 kB Progress (2): 387/586 kB | 16/17 kB Progress (2): 387/586 kB | 17 kB Progress (3): 387/586 kB | 17 kB | 16/24 kB Progress (3): 387/586 kB | 17 kB | 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar (17 kB at 35 kB/s) 2024-03-04T17:53:22.553 INFO:tasks.workunit.client.0.smithi047.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar (24 kB at 48 kB/s) 2024-03-04T17:53:22.579 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 403/586 kB Progress (1): 420/586 kB Progress (1): 436/586 kB Progress (1): 452/586 kB Progress (1): 469/586 kB Progress (2): 469/586 kB | 16/184 kB Progress (2): 469/586 kB | 33/184 kB Progress (2): 469/586 kB | 49/184 kB Progress (2): 469/586 kB | 66/184 kB Progress (2): 469/586 kB | 82/184 kB Progress (2): 485/586 kB | 82/184 kB Progress (2): 485/586 kB | 98/184 kB Progress (2): 485/586 kB | 115/184 kB Progress (2): 485/586 kB | 131/184 kB Progress (2): 485/586 kB | 147/184 kB Progress (2): 502/586 kB | 147/184 kB Progress (2): 502/586 kB | 164/184 kB Progress (2): 502/586 kB | 180/184 kB Progress (2): 502/586 kB | 184 kB Progress (2): 518/586 kB | 184 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar (184 kB at 350 kB/s) 2024-03-04T17:53:22.596 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 534/586 kB Progress (1): 551/586 kB Progress (1): 567/586 kB Progress (1): 583/586 kB Progress (1): 586 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar (586 kB at 1.1 MB/s) 2024-03-04T17:53:22.635 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:22.635 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- clean:3.2.0:clean (default-clean) @ jcksum --- 2024-03-04T17:53:22.649 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom 2024-03-04T17:53:22.685 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom (5.8 kB at 162 kB/s) 2024-03-04T17:53:22.689 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom 2024-03-04T17:53:22.728 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom (5.1 kB at 131 kB/s) 2024-03-04T17:53:22.732 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom 2024-03-04T17:53:22.769 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/43 kB Progress (1): 33/43 kB Progress (1): 43 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom (43 kB at 1.2 MB/s) 2024-03-04T17:53:22.779 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom 2024-03-04T17:53:22.810 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom (14 kB at 460 kB/s) 2024-03-04T17:53:22.820 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar 2024-03-04T17:53:22.886 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/153 kB Progress (1): 33/153 kB Progress (1): 49/153 kB Progress (1): 66/153 kB Progress (1): 82/153 kB Progress (1): 98/153 kB Progress (1): 115/153 kB Progress (1): 131/153 kB Progress (1): 147/153 kB Progress (1): 153 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar (153 kB at 2.4 MB/s) 2024-03-04T17:53:22.886 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar 2024-03-04T17:53:22.955 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/215 kB Progress (1): 33/215 kB Progress (1): 49/215 kB Progress (1): 66/215 kB Progress (1): 82/215 kB Progress (1): 98/215 kB Progress (1): 115/215 kB Progress (1): 131/215 kB Progress (1): 147/215 kB Progress (1): 164/215 kB Progress (1): 180/215 kB Progress (1): 197/215 kB Progress (1): 213/215 kB Progress (1): 215 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar (215 kB at 3.1 MB/s) 2024-03-04T17:53:23.062 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:23.063 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-03-04T17:53:23.069 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom 2024-03-04T17:53:23.098 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom (2.7 kB at 92 kB/s) 2024-03-04T17:53:23.101 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom 2024-03-04T17:53:23.135 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom (23 kB at 662 kB/s) 2024-03-04T17:53:23.140 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom 2024-03-04T17:53:23.171 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom (8.8 kB at 283 kB/s) 2024-03-04T17:53:23.175 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom 2024-03-04T17:53:23.210 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom (25 kB at 726 kB/s) 2024-03-04T17:53:23.217 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom 2024-03-04T17:53:23.244 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom (6.0 kB at 224 kB/s) 2024-03-04T17:53:23.248 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom 2024-03-04T17:53:23.273 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom (3.2 kB at 134 kB/s) 2024-03-04T17:53:23.280 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom 2024-03-04T17:53:23.304 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 612 B Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom (612 B at 24 kB/s) 2024-03-04T17:53:23.309 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom 2024-03-04T17:53:23.334 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom (2.7 kB at 114 kB/s) 2024-03-04T17:53:23.337 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom 2024-03-04T17:53:23.367 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom (14 kB at 486 kB/s) 2024-03-04T17:53:23.372 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom 2024-03-04T17:53:23.411 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom (3.2 kB at 82 kB/s) 2024-03-04T17:53:23.415 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom 2024-03-04T17:53:23.444 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom (8.4 kB at 288 kB/s) 2024-03-04T17:53:23.448 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom 2024-03-04T17:53:23.477 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom (8.4 kB at 288 kB/s) 2024-03-04T17:53:23.483 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom 2024-03-04T17:53:23.525 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom (8.0 kB at 186 kB/s) 2024-03-04T17:53:23.532 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom 2024-03-04T17:53:23.570 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom (20 kB at 519 kB/s) 2024-03-04T17:53:23.582 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom 2024-03-04T17:53:23.613 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom (5.1 kB at 159 kB/s) 2024-03-04T17:53:23.618 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom 2024-03-04T17:53:23.658 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom (31 kB at 766 kB/s) 2024-03-04T17:53:23.669 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom 2024-03-04T17:53:23.712 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom (5.1 kB at 119 kB/s) 2024-03-04T17:53:23.721 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar 2024-03-04T17:53:23.778 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/85 kB Progress (1): 32/85 kB Progress (1): 49/85 kB Progress (1): 65/85 kB Progress (1): 81/85 kB Progress (1): 85 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar (85 kB at 1.5 MB/s) 2024-03-04T17:53:23.779 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar 2024-03-04T17:53:23.779 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar 2024-03-04T17:53:23.779 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar 2024-03-04T17:53:23.780 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar 2024-03-04T17:53:23.780 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar 2024-03-04T17:53:23.816 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar (8.5 kB at 236 kB/s) 2024-03-04T17:53:23.816 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar 2024-03-04T17:53:23.819 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/269 kB Progress (2): 16/269 kB | 16/41 kB Progress (2): 33/269 kB | 16/41 kB Progress (3): 33/269 kB | 16/41 kB | 16/55 kB Progress (3): 33/269 kB | 33/41 kB | 16/55 kB Progress (3): 33/269 kB | 41 kB | 16/55 kB Progress (3): 49/269 kB | 41 kB | 16/55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar (41 kB at 1.0 MB/s) 2024-03-04T17:53:23.819 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar 2024-03-04T17:53:23.824 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 66/269 kB | 16/55 kB Progress (2): 66/269 kB | 33/55 kB Progress (2): 82/269 kB | 33/55 kB Progress (2): 98/269 kB | 33/55 kB Progress (3): 98/269 kB | 33/55 kB | 2.5 kB Progress (3): 115/269 kB | 33/55 kB | 2.5 kB Progress (3): 115/269 kB | 49/55 kB | 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar (2.5 kB at 55 kB/s) 2024-03-04T17:53:23.826 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 131/269 kB | 49/55 kB Progress (2): 131/269 kB | 55 kB Progress (2): 147/269 kB | 55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar (55 kB at 1.2 MB/s) 2024-03-04T17:53:23.839 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 164/269 kB Progress (1): 180/269 kB Progress (1): 197/269 kB Progress (1): 213/269 kB Progress (1): 229/269 kB Progress (1): 246/269 kB Progress (1): 262/269 kB Progress (1): 269 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar (269 kB at 4.4 MB/s) 2024-03-04T17:53:23.880 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/587 kB Progress (1): 33/587 kB Progress (2): 33/587 kB | 16/327 kB Progress (2): 33/587 kB | 32/327 kB Progress (2): 49/587 kB | 32/327 kB Progress (2): 49/587 kB | 49/327 kB Progress (2): 66/587 kB | 49/327 kB Progress (2): 66/587 kB | 65/327 kB Progress (2): 66/587 kB | 81/327 kB Progress (2): 82/587 kB | 81/327 kB Progress (2): 82/587 kB | 98/327 kB Progress (2): 82/587 kB | 114/327 kB Progress (2): 98/587 kB | 114/327 kB Progress (2): 98/587 kB | 130/327 kB Progress (2): 98/587 kB | 147/327 kB Progress (2): 115/587 kB | 147/327 kB Progress (2): 131/587 kB | 147/327 kB Progress (2): 131/587 kB | 163/327 kB Progress (2): 131/587 kB | 180/327 kB Progress (2): 147/587 kB | 180/327 kB Progress (2): 147/587 kB | 196/327 kB Progress (2): 164/587 kB | 196/327 kB Progress (2): 164/587 kB | 212/327 kB Progress (2): 164/587 kB | 229/327 kB Progress (2): 180/587 kB | 229/327 kB Progress (2): 180/587 kB | 245/327 kB Progress (2): 197/587 kB | 245/327 kB Progress (2): 197/587 kB | 262/327 kB Progress (2): 197/587 kB | 278/327 kB Progress (2): 213/587 kB | 278/327 kB Progress (2): 213/587 kB | 294/327 kB Progress (2): 229/587 kB | 294/327 kB Progress (2): 229/587 kB | 311/327 kB Progress (2): 229/587 kB | 327/327 kB Progress (2): 246/587 kB | 327/327 kB Progress (2): 246/587 kB | 327 kB Progress (2): 256/587 kB | 327 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar (327 kB at 3.3 MB/s) 2024-03-04T17:53:23.918 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 272/587 kB Progress (1): 289/587 kB Progress (1): 305/587 kB Progress (1): 321/587 kB Progress (1): 338/587 kB Progress (1): 354/587 kB Progress (1): 370/587 kB Progress (1): 387/587 kB Progress (1): 403/587 kB Progress (1): 420/587 kB Progress (1): 436/587 kB Progress (1): 452/587 kB Progress (1): 469/587 kB Progress (1): 485/587 kB Progress (1): 502/587 kB Progress (1): 518/587 kB Progress (1): 534/587 kB Progress (1): 551/587 kB Progress (1): 567/587 kB Progress (1): 583/587 kB Progress (1): 587 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar (587 kB at 4.3 MB/s) 2024-03-04T17:53:24.008 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-03-04T17:53:24.009 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:24.009 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-03-04T17:53:24.018 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom 2024-03-04T17:53:24.043 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom (4.7 kB at 182 kB/s) 2024-03-04T17:53:24.047 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom 2024-03-04T17:53:24.073 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom (6.4 kB at 245 kB/s) 2024-03-04T17:53:24.077 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom 2024-03-04T17:53:24.107 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/33 kB Progress (1): 32/33 kB Progress (1): 33 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom (33 kB at 1.1 MB/s) 2024-03-04T17:53:24.112 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2024-03-04T17:53:24.138 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom (14 kB at 537 kB/s) 2024-03-04T17:53:24.145 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom 2024-03-04T17:53:24.173 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 815 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom (815 B at 28 kB/s) 2024-03-04T17:53:24.177 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom 2024-03-04T17:53:24.207 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom (4.2 kB at 141 kB/s) 2024-03-04T17:53:24.211 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom 2024-03-04T17:53:24.238 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom (17 kB at 618 kB/s) 2024-03-04T17:53:24.244 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom 2024-03-04T17:53:24.271 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom (5.0 kB at 178 kB/s) 2024-03-04T17:53:24.275 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom 2024-03-04T17:53:24.300 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom (4.1 kB at 166 kB/s) 2024-03-04T17:53:24.305 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom 2024-03-04T17:53:24.330 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom (2.4 kB at 95 kB/s) 2024-03-04T17:53:24.334 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom 2024-03-04T17:53:24.359 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom (11 kB at 434 kB/s) 2024-03-04T17:53:24.364 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom 2024-03-04T17:53:24.389 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom (17 kB at 663 kB/s) 2024-03-04T17:53:24.393 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2024-03-04T17:53:24.418 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom (6.6 kB at 263 kB/s) 2024-03-04T17:53:24.423 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom 2024-03-04T17:53:24.450 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom (1.1 kB at 38 kB/s) 2024-03-04T17:53:24.453 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom 2024-03-04T17:53:24.482 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom (8.4 kB at 279 kB/s) 2024-03-04T17:53:24.486 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom 2024-03-04T17:53:24.523 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom (2.7 kB at 72 kB/s) 2024-03-04T17:53:24.528 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom 2024-03-04T17:53:24.566 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom (5.6 kB at 148 kB/s) 2024-03-04T17:53:24.571 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom 2024-03-04T17:53:24.611 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom (1.1 kB at 28 kB/s) 2024-03-04T17:53:24.618 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom 2024-03-04T17:53:24.664 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 770 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom (770 B at 17 kB/s) 2024-03-04T17:53:24.668 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom 2024-03-04T17:53:24.701 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom (6.0 kB at 183 kB/s) 2024-03-04T17:53:24.705 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom 2024-03-04T17:53:24.739 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/26 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom (26 kB at 735 kB/s) 2024-03-04T17:53:24.745 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom 2024-03-04T17:53:24.785 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom (1.2 kB at 30 kB/s) 2024-03-04T17:53:24.789 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom 2024-03-04T17:53:24.822 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom (1.3 kB at 40 kB/s) 2024-03-04T17:53:24.835 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar 2024-03-04T17:53:24.873 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar (14 kB at 356 kB/s) 2024-03-04T17:53:24.874 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar 2024-03-04T17:53:24.875 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar 2024-03-04T17:53:24.875 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar 2024-03-04T17:53:24.875 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar 2024-03-04T17:53:24.876 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar 2024-03-04T17:53:24.906 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar (4.2 kB at 128 kB/s) 2024-03-04T17:53:24.906 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/55 kB Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar 2024-03-04T17:53:24.913 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 16/55 kB | 16/122 kB Progress (2): 33/55 kB | 16/122 kB Progress (2): 33/55 kB | 33/122 kB Progress (2): 49/55 kB | 33/122 kB Progress (3): 49/55 kB | 33/122 kB | 16/27 kB Progress (3): 55 kB | 33/122 kB | 16/27 kB Progress (3): 55 kB | 49/122 kB | 16/27 kB Progress (4): 55 kB | 49/122 kB | 16/27 kB | 16/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar (55 kB at 1.4 MB/s) 2024-03-04T17:53:24.913 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar 2024-03-04T17:53:24.915 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 49/122 kB | 27 kB | 16/334 kB Progress (3): 63/122 kB | 27 kB | 16/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar (27 kB at 693 kB/s) 2024-03-04T17:53:24.915 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar 2024-03-04T17:53:24.918 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 80/122 kB | 16/334 kB Progress (2): 80/122 kB | 33/334 kB Progress (2): 96/122 kB | 33/334 kB Progress (2): 113/122 kB | 33/334 kB Progress (2): 122 kB | 33/334 kB Progress (2): 122 kB | 49/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar (122 kB at 2.8 MB/s) 2024-03-04T17:53:24.949 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 66/334 kB Progress (1): 82/334 kB Progress (1): 98/334 kB Progress (1): 115/334 kB Progress (1): 131/334 kB Progress (1): 147/334 kB Progress (2): 147/334 kB | 16/267 kB Progress (2): 147/334 kB | 33/267 kB Progress (2): 147/334 kB | 49/267 kB Progress (2): 147/334 kB | 66/267 kB Progress (2): 164/334 kB | 66/267 kB Progress (2): 164/334 kB | 82/267 kB Progress (3): 164/334 kB | 82/267 kB | 4.7 kB Progress (3): 164/334 kB | 98/267 kB | 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar (4.7 kB at 64 kB/s) 2024-03-04T17:53:24.951 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 164/334 kB | 98/267 kB | 16/23 kB Progress (3): 164/334 kB | 115/267 kB | 16/23 kB Progress (3): 180/334 kB | 115/267 kB | 16/23 kB Progress (3): 180/334 kB | 115/267 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar (23 kB at 304 kB/s) 2024-03-04T17:53:24.962 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 180/334 kB | 131/267 kB Progress (2): 180/334 kB | 147/267 kB Progress (2): 197/334 kB | 147/267 kB Progress (2): 197/334 kB | 164/267 kB Progress (2): 197/334 kB | 180/267 kB Progress (2): 197/334 kB | 197/267 kB Progress (2): 213/334 kB | 197/267 kB Progress (2): 213/334 kB | 213/267 kB Progress (2): 213/334 kB | 229/267 kB Progress (2): 213/334 kB | 246/267 kB Progress (2): 229/334 kB | 246/267 kB Progress (2): 229/334 kB | 256/267 kB Progress (2): 229/334 kB | 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar (267 kB at 3.1 MB/s) 2024-03-04T17:53:24.984 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 246/334 kB Progress (1): 256/334 kB Progress (1): 272/334 kB Progress (1): 289/334 kB Progress (1): 305/334 kB Progress (1): 321/334 kB Progress (1): 334 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar (334 kB at 3.1 MB/s) 2024-03-04T17:53:25.162 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Changes detected - recompiling the module! :source 2024-03-04T17:53:25.165 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/classes 2024-03-04T17:53:26.382 INFO:tasks.workunit.client.0.smithi047.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-03-04T17:53:26.383 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:26.383 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-03-04T17:53:26.387 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-03-04T17:53:26.387 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:26.387 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-03-04T17:53:26.395 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Changes detected - recompiling the module! :dependency 2024-03-04T17:53:26.395 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/test-classes 2024-03-04T17:53:26.639 INFO:tasks.workunit.client.0.smithi047.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-03-04T17:53:26.639 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:26.639 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-03-04T17:53:26.646 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom 2024-03-04T17:53:26.682 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom (6.2 kB at 173 kB/s) 2024-03-04T17:53:26.686 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom 2024-03-04T17:53:26.721 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom (3.5 kB at 101 kB/s) 2024-03-04T17:53:26.726 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom 2024-03-04T17:53:26.761 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom (3.3 kB at 96 kB/s) 2024-03-04T17:53:26.765 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom 2024-03-04T17:53:26.798 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom (4.1 kB at 127 kB/s) 2024-03-04T17:53:26.802 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom 2024-03-04T17:53:26.836 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom (3.3 kB at 97 kB/s) 2024-03-04T17:53:26.840 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom 2024-03-04T17:53:26.876 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom (4.5 kB at 124 kB/s) 2024-03-04T17:53:26.881 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom 2024-03-04T17:53:26.917 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom (1.8 kB at 49 kB/s) 2024-03-04T17:53:26.921 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom 2024-03-04T17:53:26.957 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom (2.2 kB at 61 kB/s) 2024-03-04T17:53:26.959 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom 2024-03-04T17:53:26.998 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom (30 kB at 773 kB/s) 2024-03-04T17:53:27.005 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom 2024-03-04T17:53:27.058 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom (1.9 kB at 36 kB/s) 2024-03-04T17:53:27.064 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom 2024-03-04T17:53:27.102 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom (5.8 kB at 153 kB/s) 2024-03-04T17:53:27.110 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom 2024-03-04T17:53:27.147 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom (2.3 kB at 63 kB/s) 2024-03-04T17:53:27.149 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom 2024-03-04T17:53:27.189 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom (22 kB at 559 kB/s) 2024-03-04T17:53:27.193 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom 2024-03-04T17:53:27.236 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/37 kB Progress (1): 33/37 kB Progress (1): 37 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom (37 kB at 871 kB/s) 2024-03-04T17:53:27.241 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2024-03-04T17:53:27.278 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom (15 kB at 412 kB/s) 2024-03-04T17:53:27.284 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom 2024-03-04T17:53:27.320 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom (8.7 kB at 240 kB/s) 2024-03-04T17:53:27.324 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom 2024-03-04T17:53:27.366 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom (27 kB at 652 kB/s) 2024-03-04T17:53:27.372 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom 2024-03-04T17:53:27.408 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom (8.1 kB at 230 kB/s) 2024-03-04T17:53:27.415 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom 2024-03-04T17:53:27.450 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom (2.2 kB at 62 kB/s) 2024-03-04T17:53:27.457 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom 2024-03-04T17:53:27.493 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom (2.6 kB at 72 kB/s) 2024-03-04T17:53:27.500 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom 2024-03-04T17:53:27.535 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom (1.5 kB at 44 kB/s) 2024-03-04T17:53:27.538 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom 2024-03-04T17:53:27.572 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom (3.1 kB at 90 kB/s) 2024-03-04T17:53:27.576 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom 2024-03-04T17:53:27.615 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom (20 kB at 538 kB/s) 2024-03-04T17:53:27.637 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom 2024-03-04T17:53:27.673 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom (6.8 kB at 188 kB/s) 2024-03-04T17:53:27.677 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom 2024-03-04T17:53:27.714 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom (14 kB at 366 kB/s) 2024-03-04T17:53:27.720 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom 2024-03-04T17:53:27.756 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom (3.0 kB at 82 kB/s) 2024-03-04T17:53:27.760 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom 2024-03-04T17:53:27.796 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom (6.8 kB at 189 kB/s) 2024-03-04T17:53:27.800 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom 2024-03-04T17:53:27.837 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom (8.4 kB at 227 kB/s) 2024-03-04T17:53:27.841 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom 2024-03-04T17:53:27.877 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom (2.1 kB at 57 kB/s) 2024-03-04T17:53:27.883 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom 2024-03-04T17:53:27.919 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom (2.2 kB at 60 kB/s) 2024-03-04T17:53:27.929 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom 2024-03-04T17:53:27.967 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom (3.0 kB at 80 kB/s) 2024-03-04T17:53:27.976 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom 2024-03-04T17:53:28.019 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom (15 kB at 348 kB/s) 2024-03-04T17:53:28.024 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom 2024-03-04T17:53:28.072 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom (15 kB at 296 kB/s) 2024-03-04T17:53:28.078 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom 2024-03-04T17:53:28.127 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom (13 kB at 274 kB/s) 2024-03-04T17:53:28.131 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom 2024-03-04T17:53:28.181 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom (4.8 kB at 96 kB/s) 2024-03-04T17:53:28.186 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom 2024-03-04T17:53:28.236 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom (6.3 kB at 126 kB/s) 2024-03-04T17:53:28.240 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom 2024-03-04T17:53:28.293 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/32 kB Progress (1): 32 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom (32 kB at 610 kB/s) 2024-03-04T17:53:28.300 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom 2024-03-04T17:53:28.345 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom (17 kB at 385 kB/s) 2024-03-04T17:53:28.350 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom 2024-03-04T17:53:28.392 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom (15 kB at 344 kB/s) 2024-03-04T17:53:28.399 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom 2024-03-04T17:53:28.438 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 750 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom (750 B at 19 kB/s) 2024-03-04T17:53:28.441 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom 2024-03-04T17:53:28.500 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom (4.8 kB at 81 kB/s) 2024-03-04T17:53:28.505 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom 2024-03-04T17:53:28.552 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 7.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom (7.9 kB at 168 kB/s) 2024-03-04T17:53:28.558 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom 2024-03-04T17:53:28.608 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom (3.0 kB at 60 kB/s) 2024-03-04T17:53:28.616 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom 2024-03-04T17:53:28.663 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom (4.2 kB at 89 kB/s) 2024-03-04T17:53:28.670 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom 2024-03-04T17:53:28.717 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom (2.1 kB at 43 kB/s) 2024-03-04T17:53:28.723 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom 2024-03-04T17:53:28.764 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom (3.5 kB at 85 kB/s) 2024-03-04T17:53:28.773 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom 2024-03-04T17:53:28.811 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom (11 kB at 288 kB/s) 2024-03-04T17:53:28.817 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom 2024-03-04T17:53:28.858 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom (13 kB at 329 kB/s) 2024-03-04T17:53:28.862 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom 2024-03-04T17:53:28.904 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/19 kB Progress (1): 19 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom (19 kB at 446 kB/s) 2024-03-04T17:53:28.909 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom 2024-03-04T17:53:28.941 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 363 B Downloaded from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom (363 B at 11 kB/s) 2024-03-04T17:53:28.945 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom 2024-03-04T17:53:28.982 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom (6.1 kB at 165 kB/s) 2024-03-04T17:53:28.986 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom 2024-03-04T17:53:29.019 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom (7.3 kB at 222 kB/s) 2024-03-04T17:53:29.022 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2024-03-04T17:53:29.051 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom (4.8 kB at 166 kB/s) 2024-03-04T17:53:29.059 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom 2024-03-04T17:53:29.093 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom (7.3 kB at 209 kB/s) 2024-03-04T17:53:29.102 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom 2024-03-04T17:53:29.141 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom (22 kB at 555 kB/s) 2024-03-04T17:53:29.145 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom 2024-03-04T17:53:29.198 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/78 kB Progress (1): 32/78 kB Progress (1): 49/78 kB Progress (1): 65/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom (78 kB at 1.5 MB/s) 2024-03-04T17:53:29.213 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom 2024-03-04T17:53:29.249 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom (23 kB at 628 kB/s) 2024-03-04T17:53:29.257 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom 2024-03-04T17:53:29.286 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom (5.6 kB at 195 kB/s) 2024-03-04T17:53:29.291 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom 2024-03-04T17:53:29.320 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom (4.3 kB at 147 kB/s) 2024-03-04T17:53:29.325 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom 2024-03-04T17:53:29.355 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom (3.2 kB at 103 kB/s) 2024-03-04T17:53:29.359 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom 2024-03-04T17:53:29.395 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/28 kB Progress (1): 28 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom (28 kB at 772 kB/s) 2024-03-04T17:53:29.401 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom 2024-03-04T17:53:29.439 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom (2.4 kB at 62 kB/s) 2024-03-04T17:53:29.455 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar 2024-03-04T17:53:29.562 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/308 kB Progress (1): 32/308 kB Progress (1): 49/308 kB Progress (1): 65/308 kB Progress (1): 81/308 kB Progress (1): 98/308 kB Progress (1): 114/308 kB Progress (1): 131/308 kB Progress (1): 147/308 kB Progress (1): 163/308 kB Progress (1): 180/308 kB Progress (1): 196/308 kB Progress (1): 212/308 kB Progress (1): 229/308 kB Progress (1): 245/308 kB Progress (1): 262/308 kB Progress (1): 278/308 kB Progress (1): 294/308 kB Progress (1): 308 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar (308 kB at 2.9 MB/s) 2024-03-04T17:53:29.563 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar 2024-03-04T17:53:29.563 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar 2024-03-04T17:53:29.564 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar 2024-03-04T17:53:29.565 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar 2024-03-04T17:53:29.565 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar 2024-03-04T17:53:29.607 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.2 kB Progress (2): 8.2 kB | 16/26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar (8.2 kB at 195 kB/s) 2024-03-04T17:53:29.607 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar 2024-03-04T17:53:29.608 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 26 kB Progress (2): 26 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar (26 kB at 574 kB/s) 2024-03-04T17:53:29.608 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar 2024-03-04T17:53:29.610 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 14 kB | 16/118 kB Progress (3): 14 kB | 16/118 kB | 16/171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar (14 kB at 289 kB/s) 2024-03-04T17:53:29.610 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar 2024-03-04T17:53:29.624 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 16/118 kB | 32/171 kB Progress (2): 33/118 kB | 32/171 kB Progress (2): 33/118 kB | 49/171 kB Progress (2): 33/118 kB | 65/171 kB Progress (2): 49/118 kB | 65/171 kB Progress (2): 49/118 kB | 81/171 kB Progress (2): 49/118 kB | 98/171 kB Progress (2): 66/118 kB | 98/171 kB Progress (2): 66/118 kB | 114/171 kB Progress (2): 82/118 kB | 114/171 kB Progress (2): 82/118 kB | 131/171 kB Progress (2): 82/118 kB | 147/171 kB Progress (2): 98/118 kB | 147/171 kB Progress (2): 98/118 kB | 163/171 kB Progress (2): 98/118 kB | 171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar (171 kB at 2.8 MB/s) 2024-03-04T17:53:29.624 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar 2024-03-04T17:53:29.626 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 115/118 kB Progress (1): 118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar (118 kB at 1.9 MB/s) 2024-03-04T17:53:29.626 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar 2024-03-04T17:53:29.661 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/146 kB Progress (1): 32/146 kB Progress (1): 49/146 kB Progress (1): 65/146 kB Progress (2): 65/146 kB | 16/136 kB Progress (2): 81/146 kB | 16/136 kB Progress (2): 98/146 kB | 16/136 kB Progress (2): 98/146 kB | 33/136 kB Progress (3): 98/146 kB | 33/136 kB | 16/61 kB Progress (3): 114/146 kB | 33/136 kB | 16/61 kB Progress (3): 114/146 kB | 33/136 kB | 32/61 kB Progress (3): 130/146 kB | 33/136 kB | 32/61 kB Progress (3): 130/146 kB | 33/136 kB | 49/61 kB Progress (3): 146 kB | 33/136 kB | 49/61 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar (146 kB at 1.5 MB/s) 2024-03-04T17:53:29.661 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar 2024-03-04T17:53:29.662 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 33/136 kB | 61 kB Progress (2): 49/136 kB | 61 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar (61 kB at 627 kB/s) 2024-03-04T17:53:29.662 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar 2024-03-04T17:53:29.678 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 66/136 kB Progress (1): 82/136 kB Progress (1): 98/136 kB Progress (2): 98/136 kB | 16/501 kB Progress (2): 115/136 kB | 16/501 kB Progress (2): 115/136 kB | 33/501 kB Progress (3): 115/136 kB | 33/501 kB | 16/58 kB Progress (3): 115/136 kB | 49/501 kB | 16/58 kB Progress (3): 131/136 kB | 49/501 kB | 16/58 kB Progress (3): 131/136 kB | 66/501 kB | 16/58 kB Progress (3): 136 kB | 66/501 kB | 16/58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar (136 kB at 1.2 MB/s) 2024-03-04T17:53:29.683 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 66/501 kB | 33/58 kB Progress (2): 82/501 kB | 33/58 kB Progress (2): 98/501 kB | 33/58 kB Progress (2): 98/501 kB | 49/58 kB Progress (2): 115/501 kB | 49/58 kB Progress (2): 115/501 kB | 58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar (58 kB at 488 kB/s) 2024-03-04T17:53:29.713 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 131/501 kB Progress (1): 147/501 kB Progress (1): 164/501 kB Progress (1): 180/501 kB Progress (1): 197/501 kB Progress (1): 213/501 kB Progress (1): 229/501 kB Progress (1): 246/501 kB Progress (1): 262/501 kB Progress (1): 279/501 kB Progress (1): 295/501 kB Progress (1): 311/501 kB Progress (1): 328/501 kB Progress (1): 344/501 kB Progress (2): 344/501 kB | 16/124 kB Progress (2): 344/501 kB | 32/124 kB Progress (2): 344/501 kB | 49/124 kB Progress (2): 344/501 kB | 65/124 kB Progress (2): 344/501 kB | 81/124 kB Progress (3): 344/501 kB | 81/124 kB | 0/2.4 MB Progress (3): 344/501 kB | 98/124 kB | 0/2.4 MB Progress (3): 344/501 kB | 114/124 kB | 0/2.4 MB Progress (3): 344/501 kB | 124 kB | 0/2.4 MB Progress (3): 344/501 kB | 124 kB | 0/2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar (124 kB at 835 kB/s) 2024-03-04T17:53:29.728 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 360/501 kB | 0/2.4 MB Progress (2): 360/501 kB | 0/2.4 MB Progress (2): 360/501 kB | 0.1/2.4 MB Progress (2): 377/501 kB | 0.1/2.4 MB Progress (2): 393/501 kB | 0.1/2.4 MB Progress (2): 393/501 kB | 0.1/2.4 MB Progress (2): 410/501 kB | 0.1/2.4 MB Progress (2): 410/501 kB | 0.1/2.4 MB Progress (2): 410/501 kB | 0.1/2.4 MB Progress (2): 426/501 kB | 0.1/2.4 MB Progress (2): 426/501 kB | 0.1/2.4 MB Progress (2): 442/501 kB | 0.1/2.4 MB Progress (2): 459/501 kB | 0.1/2.4 MB Progress (2): 459/501 kB | 0.1/2.4 MB Progress (2): 475/501 kB | 0.1/2.4 MB Progress (2): 475/501 kB | 0.2/2.4 MB Progress (2): 492/501 kB | 0.2/2.4 MB Progress (2): 501 kB | 0.2/2.4 MB Progress (2): 501 kB | 0.2/2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar (501 kB at 3.1 MB/s) 2024-03-04T17:53:29.975 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar (2.4 MB at 5.9 MB/s) 2024-03-04T17:53:30.071 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:30.072 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- jar:3.3.0:jar (default-jar) @ jcksum --- 2024-03-04T17:53:30.075 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom 2024-03-04T17:53:30.109 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom (4.5 kB at 132 kB/s) 2024-03-04T17:53:30.111 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom 2024-03-04T17:53:30.141 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom (4.9 kB at 163 kB/s) 2024-03-04T17:53:30.143 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom 2024-03-04T17:53:30.174 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/45 kB Progress (1): 32/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom (45 kB at 1.5 MB/s) 2024-03-04T17:53:30.230 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom 2024-03-04T17:53:30.230 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom (21 kB at 708 kB/s) 2024-03-04T17:53:30.231 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom 2024-03-04T17:53:30.241 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 5.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom (5.2 kB at 162 kB/s) 2024-03-04T17:53:30.244 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom 2024-03-04T17:53:30.282 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 3.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom (3.9 kB at 103 kB/s) 2024-03-04T17:53:30.286 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom 2024-03-04T17:53:30.319 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom (6.0 kB at 182 kB/s) 2024-03-04T17:53:30.322 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom 2024-03-04T17:53:30.358 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom (6.3 kB at 174 kB/s) 2024-03-04T17:53:30.361 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom 2024-03-04T17:53:30.400 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom (20 kB at 505 kB/s) 2024-03-04T17:53:30.411 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom 2024-03-04T17:53:30.454 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom (15 kB at 338 kB/s) 2024-03-04T17:53:30.460 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom 2024-03-04T17:53:30.504 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom (2.0 kB at 45 kB/s) 2024-03-04T17:53:30.509 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom 2024-03-04T17:53:30.562 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom (8.2 kB at 155 kB/s) 2024-03-04T17:53:30.566 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom 2024-03-04T17:53:30.614 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom (25 kB at 530 kB/s) 2024-03-04T17:53:30.630 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar 2024-03-04T17:53:30.682 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar (36 kB at 699 kB/s) 2024-03-04T17:53:30.682 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar 2024-03-04T17:53:30.683 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar 2024-03-04T17:53:30.683 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar 2024-03-04T17:53:30.683 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar 2024-03-04T17:53:30.684 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar 2024-03-04T17:53:30.733 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/26 kB Progress (1): 26 kB Progress (2): 26 kB | 16/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar (26 kB at 514 kB/s) 2024-03-04T17:53:30.733 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar 2024-03-04T17:53:30.746 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 16/211 kB | 16/58 kB Progress (2): 32/211 kB | 16/58 kB Progress (2): 49/211 kB | 16/58 kB Progress (2): 49/211 kB | 32/58 kB Progress (3): 49/211 kB | 32/58 kB | 0/1.0 MB Progress (3): 65/211 kB | 32/58 kB | 0/1.0 MB Progress (3): 81/211 kB | 32/58 kB | 0/1.0 MB Progress (4): 81/211 kB | 32/58 kB | 0/1.0 MB | 16/79 kB Progress (4): 81/211 kB | 32/58 kB | 0/1.0 MB | 16/79 kB Progress (4): 98/211 kB | 32/58 kB | 0/1.0 MB | 16/79 kB Progress (4): 98/211 kB | 49/58 kB | 0/1.0 MB | 16/79 kB Progress (4): 98/211 kB | 49/58 kB | 0/1.0 MB | 33/79 kB Progress (4): 114/211 kB | 49/58 kB | 0/1.0 MB | 33/79 kB Progress (4): 114/211 kB | 49/58 kB | 0/1.0 MB | 49/79 kB Progress (4): 114/211 kB | 58 kB | 0/1.0 MB | 49/79 kB Progress (4): 114/211 kB | 58 kB | 0/1.0 MB | 49/79 kB Progress (4): 114/211 kB | 58 kB | 0/1.0 MB | 66/79 kB Progress (4): 130/211 kB | 58 kB | 0/1.0 MB | 66/79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar (58 kB at 920 kB/s) 2024-03-04T17:53:30.747 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar 2024-03-04T17:53:30.750 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 147/211 kB | 0/1.0 MB | 66/79 kB Progress (3): 147/211 kB | 0/1.0 MB | 79 kB Progress (3): 163/211 kB | 0/1.0 MB | 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar (79 kB at 1.2 MB/s) 2024-03-04T17:53:30.757 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 163/211 kB | 0.1/1.0 MB Progress (2): 180/211 kB | 0.1/1.0 MB Progress (2): 196/211 kB | 0.1/1.0 MB Progress (2): 211 kB | 0.1/1.0 MB Progress (2): 211 kB | 0.1/1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar (211 kB at 2.8 MB/s) 2024-03-04T17:53:30.793 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.2/1.0 MB Progress (2): 0.2/1.0 MB | 16/116 kB Progress (2): 0.2/1.0 MB | 16/116 kB Progress (2): 0.2/1.0 MB | 32/116 kB Progress (2): 0.2/1.0 MB | 49/116 kB Progress (2): 0.2/1.0 MB | 65/116 kB Progress (2): 0.2/1.0 MB | 65/116 kB Progress (2): 0.2/1.0 MB | 81/116 kB Progress (2): 0.2/1.0 MB | 98/116 kB Progress (2): 0.2/1.0 MB | 114/116 kB Progress (2): 0.2/1.0 MB | 116 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar (116 kB at 1.1 MB/s) 2024-03-04T17:53:30.875 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (2): 0.2/1.0 MB | 16/267 kB Progress (2): 0.2/1.0 MB | 16/267 kB Progress (2): 0.2/1.0 MB | 32/267 kB Progress (2): 0.3/1.0 MB | 32/267 kB Progress (2): 0.3/1.0 MB | 49/267 kB Progress (2): 0.3/1.0 MB | 49/267 kB Progress (2): 0.3/1.0 MB | 65/267 kB Progress (2): 0.3/1.0 MB | 65/267 kB Progress (2): 0.3/1.0 MB | 81/267 kB Progress (2): 0.3/1.0 MB | 81/267 kB Progress (2): 0.3/1.0 MB | 98/267 kB Progress (2): 0.3/1.0 MB | 98/267 kB Progress (2): 0.3/1.0 MB | 114/267 kB Progress (2): 0.3/1.0 MB | 114/267 kB Progress (2): 0.3/1.0 MB | 130/267 kB Progress (2): 0.3/1.0 MB | 147/267 kB Progress (2): 0.4/1.0 MB | 147/267 kB Progress (2): 0.4/1.0 MB | 163/267 kB Progress (2): 0.4/1.0 MB | 163/267 kB Progress (2): 0.4/1.0 MB | 163/267 kB Progress (2): 0.4/1.0 MB | 163/267 kB Progress (2): 0.4/1.0 MB | 163/267 kB Progress (2): 0.4/1.0 MB | 180/267 kB Progress (2): 0.4/1.0 MB | 196/267 kB Progress (2): 0.4/1.0 MB | 196/267 kB Progress (2): 0.4/1.0 MB | 212/267 kB Progress (2): 0.5/1.0 MB | 212/267 kB Progress (2): 0.5/1.0 MB | 229/267 kB Progress (2): 0.5/1.0 MB | 229/267 kB Progress (2): 0.5/1.0 MB | 245/267 kB Progress (2): 0.5/1.0 MB | 245/267 kB Progress (2): 0.5/1.0 MB | 262/267 kB Progress (2): 0.5/1.0 MB | 267 kB Progress (2): 0.5/1.0 MB | 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar (267 kB at 1.4 MB/s) 2024-03-04T17:53:30.997 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar (1.0 MB at 3.2 MB/s) 2024-03-04T17:53:31.247 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Building jar: /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/target/jcksum-1.0.jar 2024-03-04T17:53:31.276 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:53:31.276 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] BUILD SUCCESS 2024-03-04T17:53:31.276 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:53:31.277 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Total time: 15.182 s 2024-03-04T17:53:31.277 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Finished at: 2024-03-04T17:53:31Z 2024-03-04T17:53:31.277 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:53:31.306 INFO:tasks.workunit.client.0.smithi047.stderr:+ ./mvnw test -Dtest=PutObjects 2024-03-04T17:53:31.313 INFO:tasks.workunit.client.0.smithi047.stdout:Warning: JAVA_HOME environment variable is not set. 2024-03-04T17:53:32.575 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Scanning for projects... 2024-03-04T17:53:32.674 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:32.674 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-03-04T17:53:32.674 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Building jcksum 1.0 2024-03-04T17:53:32.674 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] from pom.xml 2024-03-04T17:53:32.675 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-03-04T17:53:32.943 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:32.943 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-03-04T17:53:33.078 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-03-04T17:53:33.080 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:33.080 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-03-04T17:53:33.180 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Nothing to compile - all classes are up to date 2024-03-04T17:53:33.181 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:33.181 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-03-04T17:53:33.183 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-03-04T17:53:33.183 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:33.183 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-03-04T17:53:33.188 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Nothing to compile - all classes are up to date 2024-03-04T17:53:33.188 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:33.188 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-03-04T17:53:33.428 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider 2024-03-04T17:53:33.680 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom 2024-03-04T17:53:34.052 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/4.7 kB Progress (1): 2.8/4.7 kB Progress (1): 4.1/4.7 kB Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom (4.7 kB at 12 kB/s) 2024-03-04T17:53:34.062 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom 2024-03-04T17:53:34.125 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/2.6 kB Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom (2.6 kB at 41 kB/s) 2024-03-04T17:53:34.141 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom 2024-03-04T17:53:34.192 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom (2.8 kB at 55 kB/s) 2024-03-04T17:53:34.206 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom 2024-03-04T17:53:34.255 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/3.0 kB Progress (1): 2.8/3.0 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom (3.0 kB at 61 kB/s) 2024-03-04T17:53:34.260 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom 2024-03-04T17:53:34.308 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/3.2 kB Progress (1): 2.8/3.2 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom (3.2 kB at 67 kB/s) 2024-03-04T17:53:34.313 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom 2024-03-04T17:53:34.358 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/1.7 kB Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom (1.7 kB at 37 kB/s) 2024-03-04T17:53:34.362 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom 2024-03-04T17:53:34.408 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom (2.8 kB at 62 kB/s) 2024-03-04T17:53:34.415 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar 2024-03-04T17:53:34.478 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/27 kB Progress (1): 2.8/27 kB Progress (1): 4.1/27 kB Progress (1): 5.5/27 kB Progress (1): 6.9/27 kB Progress (1): 8.3/27 kB Progress (1): 9.7/27 kB Progress (1): 11/27 kB Progress (1): 12/27 kB Progress (1): 14/27 kB Progress (1): 15/27 kB Progress (1): 17/27 kB Progress (1): 18/27 kB Progress (1): 19/27 kB Progress (1): 21/27 kB Progress (1): 22/27 kB Progress (1): 23/27 kB Progress (1): 25/27 kB Progress (1): 26/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar (27 kB at 431 kB/s) 2024-03-04T17:53:34.481 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar 2024-03-04T17:53:34.481 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar 2024-03-04T17:53:34.482 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar 2024-03-04T17:53:34.482 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 2024-03-04T17:53:34.483 INFO:tasks.workunit.client.0.smithi047.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar 2024-03-04T17:53:34.621 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 16/189 kB Progress (1): 33/189 kB Progress (1): 49/189 kB Progress (1): 66/189 kB Progress (1): 82/189 kB Progress (1): 98/189 kB Progress (1): 115/189 kB Progress (1): 131/189 kB Progress (1): 147/189 kB Progress (1): 164/189 kB Progress (1): 180/189 kB Progress (1): 189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar (189 kB at 1.4 MB/s) 2024-03-04T17:53:34.646 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 1.4/7.7 kB Progress (1): 2.8/7.7 kB Progress (2): 2.8/7.7 kB | 1.4/18 kB Progress (2): 4.1/7.7 kB | 1.4/18 kB Progress (2): 4.1/7.7 kB | 2.8/18 kB Progress (2): 5.5/7.7 kB | 2.8/18 kB Progress (2): 5.5/7.7 kB | 4.1/18 kB Progress (2): 6.9/7.7 kB | 4.1/18 kB Progress (3): 6.9/7.7 kB | 4.1/18 kB | 16/103 kB Progress (3): 7.7 kB | 4.1/18 kB | 16/103 kB Progress (3): 7.7 kB | 5.5/18 kB | 16/103 kB Progress (4): 7.7 kB | 5.5/18 kB | 16/103 kB | 16/169 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar (7.7 kB at 46 kB/s) 2024-03-04T17:53:34.654 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (3): 6.9/18 kB | 16/103 kB | 16/169 kB Progress (3): 8.3/18 kB | 16/103 kB | 16/169 kB Progress (3): 9.7/18 kB | 16/103 kB | 16/169 kB Progress (3): 11/18 kB | 16/103 kB | 16/169 kB Progress (3): 12/18 kB | 16/103 kB | 16/169 kB Progress (3): 14/18 kB | 16/103 kB | 16/169 kB Progress (3): 15/18 kB | 16/103 kB | 16/169 kB Progress (3): 16/18 kB | 16/103 kB | 16/169 kB Progress (3): 17/18 kB | 16/103 kB | 16/169 kB Progress (3): 18 kB | 16/103 kB | 16/169 kB Progress (3): 18 kB | 33/103 kB | 16/169 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar (18 kB at 100 kB/s) 2024-03-04T17:53:34.741 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (2): 33/103 kB | 33/169 kB Progress (2): 49/103 kB | 33/169 kB Progress (2): 49/103 kB | 49/169 kB Progress (2): 66/103 kB | 49/169 kB Progress (2): 66/103 kB | 66/169 kB Progress (2): 82/103 kB | 66/169 kB Progress (2): 82/103 kB | 82/169 kB Progress (2): 98/103 kB | 82/169 kB Progress (2): 98/103 kB | 98/169 kB Progress (2): 103 kB | 98/169 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar (103 kB at 398 kB/s) 2024-03-04T17:53:34.760 INFO:tasks.workunit.client.0.smithi047.stdout:Progress (1): 115/169 kB Progress (1): 131/169 kB Progress (1): 147/169 kB Progress (1): 164/169 kB Progress (1): 169 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar (169 kB at 604 kB/s) 2024-03-04T17:53:34.797 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:53:34.798 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ------------------------------------------------------- 2024-03-04T17:53:34.798 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] T E S T S 2024-03-04T17:53:34.798 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ------------------------------------------------------- 2024-03-04T17:53:35.443 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Running io.ceph.jcksum.PutObjects 2024-03-04T17:53:35.501 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjects.java: starting test run: 2024-03-04T17:53:35.501 INFO:tasks.workunit.client.0.smithi047.stdout: AccessKey=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-03-04T17:53:35.502 INFO:tasks.workunit.client.0.smithi047.stdout: SecretKey=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-03-04T17:53:35.502 INFO:tasks.workunit.client.0.smithi047.stdout: EndpointUrl=https://smithi047.front.sepia.ceph.com:443 2024-03-04T17:53:35.598 INFO:tasks.workunit.client.0.smithi047.stderr:[main] WARN software.amazon.awssdk.http.apache.ApacheHttpClient - SSL Certificate verification is disabled. This is not a safe setting and should only be used for testing. 2024-03-04T17:53:35.698 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: Environment Variables Read 2024-03-04T17:53:36.100 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: S3 Client Initialized 2024-03-04T17:53:36.100 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: Generating File 2024-03-04T17:53:36.100 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: File Generated 2024-03-04T17:53:36.100 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: Generating File 2024-03-04T17:53:36.100 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: File Generated 2024-03-04T17:53:36.101 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: Generating File 2024-03-04T17:53:36.200 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: File Generated 2024-03-04T17:53:36.200 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: Generating File 2024-03-04T17:53:36.200 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: File Generated 2024-03-04T17:53:36.200 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: Generating File 2024-03-04T17:53:36.200 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: File Generated 2024-03-04T17:53:36.201 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: Generating File 2024-03-04T17:53:36.201 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: File Generated 2024-03-04T17:53:36.201 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: Generated Big Files 2024-03-04T17:53:36.889 INFO:tasks.workunit.client.0.smithi047.stdout:HeadBucketResponse() 2024-03-04T17:53:36.890 INFO:tasks.workunit.client.0.smithi047.stdout:sheik is ready 2024-03-04T17:53:36.890 INFO:tasks.workunit.client.0.smithi047.stdout:DEBUG: Test Bucket Created 2024-03-04T17:53:37.002 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileNoCksum called with file-0b 2024-03-04T17:53:37.303 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileNoCksum called with file-8b 2024-03-04T17:53:37.603 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileNoCksum called with file-200b 2024-03-04T17:53:37.904 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileNoCksum called with file-21983b 2024-03-04T17:53:38.205 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileNoCksum called with file-5519b 2024-03-04T17:53:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileNoCksum called with file-204329b 2024-03-04T17:53:39.007 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileNoCksum called with file-256k 2024-03-04T17:53:39.508 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileNoCksum called with file-1m 2024-03-04T17:53:40.710 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileNoCksum called with file-1038757b 2024-03-04T17:53:41.813 INFO:tasks.workunit.client.0.smithi047.stdout:mpuObjectFromFileNoCksum called with file-5m 2024-03-04T17:53:48.728 INFO:tasks.workunit.client.0.smithi047.stdout:mpuObjectFromFileNoCksum called with file-10m 2024-03-04T17:53:58.549 INFO:tasks.workunit.client.0.smithi047.stdout:mpuObjectFromFileNoCksum called with file-100m 2024-03-04T17:55:12.909 INFO:tasks.workunit.client.0.smithi047.stdout:arg: file-0b 2024-03-04T17:55:12.909 INFO:tasks.workunit.client.0.smithi047.stdout:arg: file-8b 2024-03-04T17:55:12.910 INFO:tasks.workunit.client.0.smithi047.stdout:arg: file-200b 2024-03-04T17:55:12.911 INFO:tasks.workunit.client.0.smithi047.stdout:arg: file-21983b 2024-03-04T17:55:12.913 INFO:tasks.workunit.client.0.smithi047.stdout:arg: file-5519b 2024-03-04T17:55:12.914 INFO:tasks.workunit.client.0.smithi047.stdout:arg: file-204329b 2024-03-04T17:55:12.915 INFO:tasks.workunit.client.0.smithi047.stdout:arg: file-256k 2024-03-04T17:55:12.916 INFO:tasks.workunit.client.0.smithi047.stdout:arg: file-1m 2024-03-04T17:55:12.918 INFO:tasks.workunit.client.0.smithi047.stdout:arg: file-1038757b 2024-03-04T17:55:12.952 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileCksum called with file-0b 2024-03-04T17:55:13.152 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjectResponse 2024-03-04T17:55:13.252 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileCksum called with file-8b 2024-03-04T17:55:13.352 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjectResponse 2024-03-04T17:55:13.453 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileCksum called with file-200b 2024-03-04T17:55:13.653 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjectResponse 2024-03-04T17:55:13.753 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileCksum called with file-21983b 2024-03-04T17:55:13.853 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjectResponse 2024-03-04T17:55:13.953 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileCksum called with file-5519b 2024-03-04T17:55:14.153 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjectResponse 2024-03-04T17:55:14.254 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileCksum called with file-204329b 2024-03-04T17:55:14.554 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjectResponse 2024-03-04T17:55:14.754 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileCksum called with file-256k 2024-03-04T17:55:15.055 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjectResponse 2024-03-04T17:55:15.255 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileCksum called with file-1m 2024-03-04T17:55:16.156 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjectResponse 2024-03-04T17:55:16.356 INFO:tasks.workunit.client.0.smithi047.stdout:putObjectFromFileCksum called with file-1038757b 2024-03-04T17:55:17.458 INFO:tasks.workunit.client.0.smithi047.stdout:PutObjectResponse 2024-03-04T17:55:17.759 INFO:tasks.workunit.client.0.smithi047.stdout:mpuObjectFromFileCksum called with file-5m 2024-03-04T17:55:22.765 INFO:tasks.workunit.client.0.smithi047.stdout:mpuObjectFromFileCksum called with file-10m 2024-03-04T17:55:32.277 INFO:tasks.workunit.client.0.smithi047.stdout:mpuObjectFromFileCksum called with file-100m 2024-03-04T17:57:03.015 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 207.6 s -- in io.ceph.jcksum.PutObjects 2024-03-04T17:57:03.039 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:57:03.039 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Results: 2024-03-04T17:57:03.039 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:57:03.039 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0 2024-03-04T17:57:03.039 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] 2024-03-04T17:57:03.041 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:57:03.041 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] BUILD SUCCESS 2024-03-04T17:57:03.041 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:57:03.042 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Total time: 03:30 min 2024-03-04T17:57:03.042 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] Finished at: 2024-03-04T17:57:03Z 2024-03-04T17:57:03.042 INFO:tasks.workunit.client.0.smithi047.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:57:03.072 INFO:tasks.workunit.client.0.smithi047.stderr:+ exit 2024-03-04T17:57:03.072 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:57:03.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:57:03.127 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh'] on client.0... 2024-03-04T17:57:03.127 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T17:57:03.509 DEBUG:teuthology.parallel:result is None 2024-03-04T17:57:03.510 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:57:03.523 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:57:03.523 DEBUG:teuthology.orchestra.run.smithi047:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:57:03.572 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:57:03.572 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T17:57:03.583 INFO:tasks.workunit:Pulling workunits from ref 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:57:03.583 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T17:57:03.583 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:57:03.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:57:03.621 INFO:teuthology.orchestra.run.smithi047.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T17:57:03.621 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:57:03.669 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T17:57:03.669 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T17:57:03.721 INFO:tasks.workunit:timeout=3h 2024-03-04T17:57:03.721 INFO:tasks.workunit:cleanup=True 2024-03-04T17:57:03.721 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/pritha-srivastava/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:57:03.771 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T17:58:24.549 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to '3aec390336fdfdc8f927818b718cfb0361bd47c8'. 2024-03-04T17:58:24.549 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:58:24.549 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T17:58:24.549 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T17:58:24.549 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T17:58:24.550 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at 3aec390336f qa/d4n: Start redis up before rgw 2024-03-04T17:58:24.556 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T17:58:24.605 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T17:58:24.608 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:58:24.608 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T17:58:24.672 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T17:58:24.727 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T17:58:24.775 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:58:24.777 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:58:24.777 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T17:58:24.823 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:58:24.826 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:58:24.826 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T17:58:24.873 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_s3_mp_reupload.sh on client.0... 2024-03-04T17:58:24.874 INFO:tasks.workunit:Running workunit rgw/test_rgw_s3_mp_reupload.sh... 2024-03-04T17:58:24.874 DEBUG:teuthology.orchestra.run.smithi047:workunit test rgw/test_rgw_s3_mp_reupload.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh 2024-03-04T17:58:24.928 INFO:tasks.workunit.client.0.smithi047.stdout:INFO: fully qualified domain name: smithi047.front.sepia.ceph.com 2024-03-04T17:58:24.928 INFO:tasks.workunit.client.0.smithi047.stdout:== TESTING *ABORTING* MULTIPART UPLOAD WITH RE-UPLOADS == 2024-03-04T17:58:24.928 INFO:tasks.workunit.client.0.smithi047.stdout:== TESTING VERSIONED BUCKET == 2024-03-04T17:58:28.329 INFO:tasks.workunit.client.0.smithi047.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-03-04T17:58:28.557 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting pip 2024-03-04T17:58:28.564 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-03-04T17:58:28.603 INFO:tasks.workunit.client.0.smithi047.stdout:Installing collected packages: pip 2024-03-04T17:58:28.603 INFO:tasks.workunit.client.0.smithi047.stdout: Attempting uninstall: pip 2024-03-04T17:58:28.603 INFO:tasks.workunit.client.0.smithi047.stdout: Found existing installation: pip 22.0.2 2024-03-04T17:58:28.873 INFO:tasks.workunit.client.0.smithi047.stdout: Uninstalling pip-22.0.2: 2024-03-04T17:58:28.879 INFO:tasks.workunit.client.0.smithi047.stdout: Successfully uninstalled pip-22.0.2 2024-03-04T17:58:29.848 INFO:tasks.workunit.client.0.smithi047.stdout:Successfully installed pip-24.0 2024-03-04T17:58:30.861 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting boto3 2024-03-04T17:58:30.863 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached boto3-1.34.54-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T17:58:31.408 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting botocore<1.35.0,>=1.34.54 (from boto3) 2024-03-04T17:58:31.409 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached botocore-1.34.54-py3-none-any.whl.metadata (5.7 kB) 2024-03-04T17:58:31.477 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-03-04T17:58:31.478 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-03-04T17:58:31.548 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-03-04T17:58:31.549 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached s3transfer-0.10.0-py3-none-any.whl.metadata (1.7 kB) 2024-03-04T17:58:31.647 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:58:31.650 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-03-04T17:58:31.806 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting urllib3<2.1,>=1.25.4 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:58:31.807 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T17:58:31.982 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:58:31.984 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-03-04T17:58:32.001 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached boto3-1.34.54-py3-none-any.whl (139 kB) 2024-03-04T17:58:32.003 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached botocore-1.34.54-py3-none-any.whl (12.0 MB) 2024-03-04T17:58:32.030 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-03-04T17:58:32.032 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached s3transfer-0.10.0-py3-none-any.whl (82 kB) 2024-03-04T17:58:32.033 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-03-04T17:58:32.035 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached urllib3-2.0.7-py3-none-any.whl (124 kB) 2024-03-04T17:58:32.037 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-03-04T17:58:32.141 INFO:tasks.workunit.client.0.smithi047.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-03-04T17:58:33.331 INFO:tasks.workunit.client.0.smithi047.stdout:Successfully installed boto3-1.34.54 botocore-1.34.54 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.0 six-1.16.0 urllib3-2.0.7 2024-03-04T17:58:33.671 INFO:tasks.workunit.client.0.smithi047.stdout:INFO: creating user 2024-03-04T17:58:34.248 INFO:tasks.workunit.client.0.smithi047.stdout:{ 2024-03-04T17:58:34.248 INFO:tasks.workunit.client.0.smithi047.stdout: "user_id": "testid", 2024-03-04T17:58:34.248 INFO:tasks.workunit.client.0.smithi047.stdout: "display_name": "M. Tester", 2024-03-04T17:58:34.248 INFO:tasks.workunit.client.0.smithi047.stdout: "email": "tester@ceph.com", 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "suspended": 0, 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "max_buckets": 1000, 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "subusers": [], 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "keys": [ 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: { 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "user": "testid", 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "access_key": "0555b35654ad1656d804", 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==", 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "active": true 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: } 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: ], 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "swift_keys": [], 2024-03-04T17:58:34.249 INFO:tasks.workunit.client.0.smithi047.stdout: "caps": [], 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "default_placement": "", 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "default_storage_class": "", 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "placement_tags": [], 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "bucket_quota": { 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "enabled": false, 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size": -1, 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "max_objects": -1 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "user_quota": { 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "enabled": false, 2024-03-04T17:58:34.250 INFO:tasks.workunit.client.0.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:58:34.251 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size": -1, 2024-03-04T17:58:34.251 INFO:tasks.workunit.client.0.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:58:34.251 INFO:tasks.workunit.client.0.smithi047.stdout: "max_objects": -1 2024-03-04T17:58:34.251 INFO:tasks.workunit.client.0.smithi047.stdout: }, 2024-03-04T17:58:34.251 INFO:tasks.workunit.client.0.smithi047.stdout: "temp_url_keys": [], 2024-03-04T17:58:34.251 INFO:tasks.workunit.client.0.smithi047.stdout: "type": "rgw", 2024-03-04T17:58:34.251 INFO:tasks.workunit.client.0.smithi047.stdout: "mfa_ids": [] 2024-03-04T17:58:34.251 INFO:tasks.workunit.client.0.smithi047.stdout:} 2024-03-04T17:58:34.251 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-04T17:58:44.072 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:44.073 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:58:44.225 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:44.225 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:58:44.317 INFO:tasks.workunit.client.0.smithi047.stderr:5+0 records in 2024-03-04T17:58:44.318 INFO:tasks.workunit.client.0.smithi047.stderr:5+0 records out 2024-03-04T17:58:44.318 INFO:tasks.workunit.client.0.smithi047.stderr:5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0215616 s, 243 MB/s 2024-03-04T17:58:44.323 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:44.323 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:58:44.457 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:44.457 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:58:46.907 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:46.907 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:58:49.449 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:49.449 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:58:51.964 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:51.964 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:58:54.442 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:54.442 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:58:56.942 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:56.942 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:58:59.486 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:59.486 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:59:02.028 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:59:02.028 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:59:04.562 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:59:04.562 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:59:07.062 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:59:07.062 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:59:09.616 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:59:09.617 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:59:12.067 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:59:12.067 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:59:14.601 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:59:14.601 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:59:17.108 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi047.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:59:17.108 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T17:59:17.316 INFO:tasks.workunit.client.0.smithi047.stdout:bucket nams is reupload-bkt-128242 2024-03-04T17:59:17.316 INFO:tasks.workunit.client.0.smithi047.stdout:endpoint is https://smithi047.front.sepia.ceph.com:443 2024-03-04T17:59:17.316 INFO:tasks.workunit.client.0.smithi047.stdout:start UploadId=2~C3w29pBhIVuxKtj0Xee8Pa7vJ5bPRkq 2024-03-04T17:59:17.316 INFO:tasks.workunit.client.0.smithi047.stdout:phase 1 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.316 INFO:tasks.workunit.client.0.smithi047.stdout:phase 1 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 2} 2024-03-04T17:59:17.316 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.316 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:phase 2 uploaded part {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1} 2024-03-04T17:59:17.317 INFO:tasks.workunit.client.0.smithi047.stdout:aborting multipart upload, parts=[{'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 1}, {'ETag': '"f78c3544e95ad07b6876d5544d134a1f"', 'PartNumber': 2}] 2024-03-04T17:59:18.824 INFO:tasks.workunit.client.0.smithi047.stdout:radosgw-admin bucket radoslist: 2024-03-04T17:59:19.133 INFO:tasks.workunit.client.0.smithi047.stdout:rados ls: 2024-03-04T17:59:19.148 INFO:tasks.workunit.client.0.smithi047.stdout:== PASSED == 2024-03-04T17:59:19.150 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:59:19.150 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:59:19.208 INFO:tasks.workunit:Stopping ['rgw/test_rgw_s3_mp_reupload.sh'] on client.0... 2024-03-04T17:59:19.208 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T17:59:19.648 DEBUG:teuthology.parallel:result is None 2024-03-04T17:59:19.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:59:19.662 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:59:19.662 DEBUG:teuthology.orchestra.run.smithi047:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:59:19.707 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:59:19.707 INFO:teuthology.run_tasks:Running task ragweed... 2024-03-04T17:59:19.725 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}} 2024-03-04T17:59:19.725 INFO:tasks.ragweed:Downloading ragweed... 2024-03-04T17:59:19.726 INFO:tasks.ragweed:Using branch 'wip-rgw-d4n' for ragweed 2024-03-04T17:59:19.726 DEBUG:teuthology.orchestra.run.smithi047:> git clone -b wip-rgw-d4n https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-03-04T17:59:19.755 INFO:teuthology.orchestra.run.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-03-04T17:59:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:fatal: Remote branch wip-rgw-d4n not found in upstream origin 2024-03-04T17:59:19.808 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-03-04T17:59:19.809 INFO:tasks.ragweed:Using branch 'ceph-master' for ragweed 2024-03-04T17:59:19.809 DEBUG:teuthology.orchestra.run.smithi047:> git clone -b ceph-master https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-03-04T17:59:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-03-04T17:59:20.036 INFO:tasks.ragweed:Creating rgw users... 2024-03-04T17:59:20.036 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2024-03-04T17:59:20.037 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key RRNEBMPDKTWKEMLCMFII --secret +7V6TD71KVVAJkv/DWNXZil68ztnGq4Wr55aMlYgQHAxmgLAtgv9SQ== --email ragweed.client.0+test@test.test --system false 2024-03-04T17:59:20.127 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:59:20.127 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:59:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.168+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 20 realm 2024-03-04T17:59:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:59:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:59:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.172+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.176+0000 7f39b2355980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:59:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.176+0000 7f39b2355980 20 searching for the correct realm 2024-03-04T17:59:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.180+0000 7f39af66c640 0 -- 172.21.15.47:0/3698377265 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f3998103040 msgr2=0x7f39981178a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:59:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.188+0000 7f39afe6d640 0 -- 172.21.15.47:0/3698377265 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f39a0080c20 msgr2=0x7f39a00599c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:59:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:59:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:59:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:59:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:59:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:59:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:59:20.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:59:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:59:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 zone default found 2024-03-04T17:59:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 4 Realm: () 2024-03-04T17:59:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:59:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:59:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:59:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 zonegroup default 2024-03-04T17:59:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.196+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.200+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.200+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.200+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.200+0000 7f39b2355980 20 started sync module instance, tier type = 2024-03-04T17:59:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.200+0000 7f39b2355980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:59:20.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.216+0000 7f39b2355980 20 add_watcher() i=0 2024-03-04T17:59:20.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.216+0000 7f39b2355980 20 add_watcher() i=1 2024-03-04T17:59:20.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.216+0000 7f39b2355980 20 add_watcher() i=2 2024-03-04T17:59:20.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.216+0000 7f39b2355980 20 add_watcher() i=3 2024-03-04T17:59:20.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.216+0000 7f39b2355980 20 add_watcher() i=4 2024-03-04T17:59:20.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.216+0000 7f39b2355980 20 add_watcher() i=5 2024-03-04T17:59:20.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.216+0000 7f39b2355980 20 add_watcher() i=6 2024-03-04T17:59:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.220+0000 7f39b2355980 20 add_watcher() i=7 2024-03-04T17:59:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.220+0000 7f39b2355980 2 all 8 watchers are set, enabling cache 2024-03-04T17:59:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f3865fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:59:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:59:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:59:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 5 note: GC not initialized 2024-03-04T17:59:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f3864fe1640 20 reqs_thread_entry: start 2024-03-04T17:59:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 init_complete bucket index max shards: 11 2024-03-04T17:59:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 Filter name: none 2024-03-04T17:59:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f37de7fc640 20 reqs_thread_entry: start 2024-03-04T17:59:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-03-04T17:59:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2024-03-04T17:59:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-03-04T17:59:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2024-03-04T17:59:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2024-03-04T17:59:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-03-04T17:59:20.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 cache get: name=default.rgw.meta+users.keys+RRNEBMPDKTWKEMLCMFII : miss 2024-03-04T17:59:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 cache put: name=default.rgw.meta+users.keys+RRNEBMPDKTWKEMLCMFII info.flags=0x0 2024-03-04T17:59:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 adding default.rgw.meta+users.keys+RRNEBMPDKTWKEMLCMFII to cache LRU end 2024-03-04T17:59:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 cache get: name=default.rgw.meta+users.keys+RRNEBMPDKTWKEMLCMFII : hit (negative entry) 2024-03-04T17:59:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.224+0000 7f39b2355980 10 cache get: name=default.rgw.meta+users.keys+RRNEBMPDKTWKEMLCMFII : hit (negative entry) 2024-03-04T17:59:20.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.228+0000 7f39b2355980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-03-04T17:59:20.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.228+0000 7f39b2355980 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-03-04T17:59:20.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.228+0000 7f39b2355980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-03-04T17:59:20.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.228+0000 7f388effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791404 cookie 94363788790544 notifier 5877 bl.length()=642 2024-03-04T17:59:20.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.228+0000 7f388effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-03-04T17:59:20.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.228+0000 7f388effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-03-04T17:59:20.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.228+0000 7f39b2355980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-03-04T17:59:20.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.228+0000 7f39b2355980 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-03-04T17:59:20.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.228+0000 7f39b2355980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-03-04T17:59:20.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.232+0000 7f388effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791309 cookie 94363788782144 notifier 5877 bl.length()=202 2024-03-04T17:59:20.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.232+0000 7f388effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-03-04T17:59:20.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.232+0000 7f388effd640 10 rgw watcher librados: moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-03-04T17:59:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.232+0000 7f39b2355980 10 cache put: name=default.rgw.meta+users.keys+RRNEBMPDKTWKEMLCMFII info.flags=0x7 2024-03-04T17:59:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.232+0000 7f39b2355980 10 moving default.rgw.meta+users.keys+RRNEBMPDKTWKEMLCMFII to cache LRU end 2024-03-04T17:59:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.232+0000 7f39b2355980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:RRNEBMPDKTWKEMLCMFII, ofs0, ns] 2024-03-04T17:59:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.232+0000 7f388effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791310 cookie 94363788794976 notifier 5877 bl.length()=190 2024-03-04T17:59:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.232+0000 7f388effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+RRNEBMPDKTWKEMLCMFII info.flags=0x7 2024-03-04T17:59:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.232+0000 7f388effd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+RRNEBMPDKTWKEMLCMFII to cache LRU end 2024-03-04T17:59:20.232 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "user_id": "ragweed.client.0", 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "display_name": "Mr. ragweed.client.0", 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "email": "ragweed.client.0+test@test.test", 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "suspended": 0, 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "max_buckets": 1000, 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "subusers": [], 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "keys": [ 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "user": "ragweed.client.0", 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "access_key": "RRNEBMPDKTWKEMLCMFII", 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "secret_key": "+7V6TD71KVVAJkv/DWNXZil68ztnGq4Wr55aMlYgQHAxmgLAtgv9SQ==", 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: "active": true 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T17:59:20.233 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "swift_keys": [], 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": [], 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "default_placement": "", 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "default_storage_class": "", 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "placement_tags": [], 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "bucket_quota": { 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T17:59:20.234 INFO:teuthology.orchestra.run.smithi047.stdout: "user_quota": { 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: "temp_url_keys": [], 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "rgw", 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: "mfa_ids": [] 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-04T17:59:20.235 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:59:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.236+0000 7f3865fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:59:20.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.240+0000 7f39b2355980 20 remove_watcher() i=0 2024-03-04T17:59:20.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.240+0000 7f39b2355980 2 removed watcher, disabling cache 2024-03-04T17:59:20.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.244+0000 7f39b2355980 20 remove_watcher() i=1 2024-03-04T17:59:20.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.244+0000 7f39b2355980 20 remove_watcher() i=2 2024-03-04T17:59:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.248+0000 7f39b2355980 20 remove_watcher() i=3 2024-03-04T17:59:20.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.248+0000 7f39b2355980 20 remove_watcher() i=4 2024-03-04T17:59:20.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.252+0000 7f39b2355980 20 remove_watcher() i=5 2024-03-04T17:59:20.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.252+0000 7f39b2355980 20 remove_watcher() i=6 2024-03-04T17:59:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.256+0000 7f39b2355980 20 remove_watcher() i=7 2024-03-04T17:59:20.269 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2024-03-04T17:59:20.269 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key WIJRHVIFQUKMQINMYKYW --secret CNQNdgJ/FtFgpDk9C/sko0ayjDvqAvPFQ4RxXoBALqIq78ekvxBa3w== --email sysuser.client.0+test@test.test --system true 2024-03-04T17:59:20.310 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:59:20.310 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:59:20.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.352+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.352+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.352+0000 7f7312c61980 20 realm 2024-03-04T17:59:20.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.352+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.356+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.356+0000 7f7312c61980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:59:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.356+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.356+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.356+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.356+0000 7f7312c61980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:59:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.356+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.356+0000 7f7312c61980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:59:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.356+0000 7f7312c61980 20 searching for the correct realm 2024-03-04T17:59:20.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.364+0000 7f7310f7a640 0 -- 172.21.15.47:0/4196835084 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f730404c030 msgr2=0x7f7304054e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:59:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.368+0000 7f7310779640 0 -- 172.21.15.47:0/4196835084 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f7308035040 msgr2=0x7f7308035930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:59:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.376+0000 7f7312c61980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T17:59:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.376+0000 7f7312c61980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T17:59:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.376+0000 7f7312c61980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:59:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.376+0000 7f7312c61980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:59:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.376+0000 7f7312c61980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:59:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.376+0000 7f7312c61980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:59:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.376+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.376+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.376+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:59:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:59:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 zone default found 2024-03-04T17:59:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 4 Realm: () 2024-03-04T17:59:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T17:59:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T17:59:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:59:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 zonegroup default 2024-03-04T17:59:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 started sync module instance, tier type = 2024-03-04T17:59:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.380+0000 7f7312c61980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T17:59:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.400+0000 7f7312c61980 20 add_watcher() i=0 2024-03-04T17:59:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.400+0000 7f7312c61980 20 add_watcher() i=1 2024-03-04T17:59:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.400+0000 7f7312c61980 20 add_watcher() i=2 2024-03-04T17:59:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.400+0000 7f7312c61980 20 add_watcher() i=3 2024-03-04T17:59:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.400+0000 7f7312c61980 20 add_watcher() i=4 2024-03-04T17:59:20.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.400+0000 7f7312c61980 20 add_watcher() i=5 2024-03-04T17:59:20.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.400+0000 7f7312c61980 20 add_watcher() i=6 2024-03-04T17:59:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.400+0000 7f7312c61980 20 add_watcher() i=7 2024-03-04T17:59:20.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.400+0000 7f7312c61980 2 all 8 watchers are set, enabling cache 2024-03-04T17:59:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f71c6fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:59:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:59:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:59:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 5 note: GC not initialized 2024-03-04T17:59:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f71c5fdb640 20 reqs_thread_entry: start 2024-03-04T17:59:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 20 init_complete bucket index max shards: 11 2024-03-04T17:59:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 20 Filter name: none 2024-03-04T17:59:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f71377fe640 20 reqs_thread_entry: start 2024-03-04T17:59:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-03-04T17:59:20.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2024-03-04T17:59:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-03-04T17:59:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2024-03-04T17:59:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2024-03-04T17:59:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-03-04T17:59:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 10 cache get: name=default.rgw.meta+users.keys+WIJRHVIFQUKMQINMYKYW : miss 2024-03-04T17:59:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.404+0000 7f7312c61980 20 rados->read ofs=0 len=0 2024-03-04T17:59:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f7312c61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:59:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f7312c61980 10 cache put: name=default.rgw.meta+users.keys+WIJRHVIFQUKMQINMYKYW info.flags=0x0 2024-03-04T17:59:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f7312c61980 10 adding default.rgw.meta+users.keys+WIJRHVIFQUKMQINMYKYW to cache LRU end 2024-03-04T17:59:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f7312c61980 10 cache get: name=default.rgw.meta+users.keys+WIJRHVIFQUKMQINMYKYW : hit (negative entry) 2024-03-04T17:59:20.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f7312c61980 10 cache get: name=default.rgw.meta+users.keys+WIJRHVIFQUKMQINMYKYW : hit (negative entry) 2024-03-04T17:59:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f7312c61980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-03-04T17:59:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f7312c61980 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-03-04T17:59:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f7312c61980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-03-04T17:59:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f71e77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791310 cookie 93841290892352 notifier 5898 bl.length()=642 2024-03-04T17:59:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f71e77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-03-04T17:59:20.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.408+0000 7f71e77fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-03-04T17:59:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.412+0000 7f7312c61980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-03-04T17:59:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.412+0000 7f7312c61980 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-03-04T17:59:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.412+0000 7f7312c61980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-03-04T17:59:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.416+0000 7f71e77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791405 cookie 93841290903328 notifier 5898 bl.length()=202 2024-03-04T17:59:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.416+0000 7f71e77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-03-04T17:59:20.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.416+0000 7f71e77fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-03-04T17:59:20.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.420+0000 7f7312c61980 10 cache put: name=default.rgw.meta+users.keys+WIJRHVIFQUKMQINMYKYW info.flags=0x7 2024-03-04T17:59:20.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.420+0000 7f7312c61980 10 moving default.rgw.meta+users.keys+WIJRHVIFQUKMQINMYKYW to cache LRU end 2024-03-04T17:59:20.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.420+0000 7f7312c61980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:WIJRHVIFQUKMQINMYKYW, ofs0, ns] 2024-03-04T17:59:20.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.420+0000 7f71e6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791311 cookie 93841290906880 notifier 5898 bl.length()=190 2024-03-04T17:59:20.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.420+0000 7f71e6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+WIJRHVIFQUKMQINMYKYW info.flags=0x7 2024-03-04T17:59:20.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.420+0000 7f71e6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+WIJRHVIFQUKMQINMYKYW to cache LRU end 2024-03-04T17:59:20.418 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-04T17:59:20.418 INFO:teuthology.orchestra.run.smithi047.stdout: "user_id": "sysuser.client.0", 2024-03-04T17:59:20.418 INFO:teuthology.orchestra.run.smithi047.stdout: "display_name": "Mr. sysuser.client.0", 2024-03-04T17:59:20.418 INFO:teuthology.orchestra.run.smithi047.stdout: "email": "sysuser.client.0+test@test.test", 2024-03-04T17:59:20.418 INFO:teuthology.orchestra.run.smithi047.stdout: "suspended": 0, 2024-03-04T17:59:20.418 INFO:teuthology.orchestra.run.smithi047.stdout: "max_buckets": 1000, 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "subusers": [], 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "keys": [ 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "user": "sysuser.client.0", 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "access_key": "WIJRHVIFQUKMQINMYKYW", 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "secret_key": "CNQNdgJ/FtFgpDk9C/sko0ayjDvqAvPFQ4RxXoBALqIq78ekvxBa3w==", 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "active": true 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "swift_keys": [], 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": [], 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "system": true, 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "default_placement": "", 2024-03-04T17:59:20.419 INFO:teuthology.orchestra.run.smithi047.stdout: "default_storage_class": "", 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "placement_tags": [], 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "bucket_quota": { 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "user_quota": { 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T17:59:20.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T17:59:20.421 INFO:teuthology.orchestra.run.smithi047.stdout: "temp_url_keys": [], 2024-03-04T17:59:20.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "rgw", 2024-03-04T17:59:20.421 INFO:teuthology.orchestra.run.smithi047.stdout: "mfa_ids": [] 2024-03-04T17:59:20.421 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-04T17:59:20.421 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:59:20.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.424+0000 7f71c6fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:59:20.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.428+0000 7f7312c61980 20 remove_watcher() i=0 2024-03-04T17:59:20.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.428+0000 7f7312c61980 2 removed watcher, disabling cache 2024-03-04T17:59:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.428+0000 7f7312c61980 20 remove_watcher() i=1 2024-03-04T17:59:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.432+0000 7f7312c61980 20 remove_watcher() i=2 2024-03-04T17:59:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.432+0000 7f7312c61980 20 remove_watcher() i=3 2024-03-04T17:59:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.436+0000 7f7312c61980 20 remove_watcher() i=4 2024-03-04T17:59:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.436+0000 7f7312c61980 20 remove_watcher() i=5 2024-03-04T17:59:20.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.440+0000 7f7312c61980 20 remove_watcher() i=6 2024-03-04T17:59:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T17:59:20.444+0000 7f7312c61980 20 remove_watcher() i=7 2024-03-04T17:59:20.454 INFO:tasks.ragweed:Configuring ragweed... 2024-03-04T17:59:20.454 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:59:20.454 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/archive/ragweed.client.0.conf 2024-03-04T17:59:20.504 INFO:tasks.ragweed:Configuring boto... 2024-03-04T17:59:20.505 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T17:59:20.505 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/boto.cfg 2024-03-04T17:59:20.556 DEBUG:teuthology.orchestra.run.smithi047:ragweed tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw' 2024-03-04T17:59:20.730 INFO:teuthology.orchestra.run.smithi047.stdout:GLOB sdist-make: /home/ubuntu/cephtest/ragweed.client.0/setup.py 2024-03-04T17:59:20.919 INFO:teuthology.orchestra.run.smithi047.stdout:py create: /home/ubuntu/cephtest/ragweed.client.0/.tox/py 2024-03-04T17:59:21.462 INFO:teuthology.orchestra.run.smithi047.stdout:py installdeps: -rrequirements.txt 2024-03-04T17:59:25.520 INFO:teuthology.orchestra.run.smithi047.stdout:py inst: /home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip 2024-03-04T17:59:28.360 INFO:teuthology.orchestra.run.smithi047.stdout:py installed: appdirs==1.4.4,asyncssh==2.5.0,attrs==21.2.0,Babel==2.8.0,bcrypt==3.2.0,beautifulsoup4==4.10.0,beniget==0.4.1,blinker==1.4,boto==2.49.0,Brotli==1.0.9,cachetools==5.0.0,ceph==1.0.0,ceph-volume==1.0.0,cephfs==2.0.0,certifi==2020.6.20,chardet==4.0.0,cheroot==8.5.2+ds1,CherryPy==18.6.1,command-not-found==0.3,configobj==5.0.6,cryptography==3.4.8,cycler==0.11.0,dbus-python==1.2.18,decorator==4.4.2,distlib==0.3.4,distro==1.7.0,distro-info===1.1build1,docopt==0.6.2,execnet==1.9.0,filelock==3.6.0,fonttools==4.29.1,fs==2.4.12,gast==0.5.2,gevent==21.8.0,google-auth==1.5.1,greenlet==1.1.2,html5lib==1.1,httplib2==0.20.2,idna==3.3,importlib-metadata==4.6.4,iniconfig==1.1.1,isodate==0.6.1,jaraco.classes==3.2.1,jaraco.collections==3.4.0,jaraco.functools==3.4.0,jaraco.text==3.6.0,jeepney==0.7.1,Jinja2==3.0.3,joblib==0.17.0,keyring==23.5.0,kiwisolver==1.3.2,kubernetes==12.0.1,language-selector==0.1,launchpadlib==1.10.16,lazr.restfulclient==0.14.4,lazr.uri==1.0.6,logutils==0.3.3,lxml==4.8.0,lz4==3.1.3+dfsg,Mako==1.1.3,MarkupSafe==2.0.1,matplotlib==3.5.1,more-itertools==8.10.0,mpmath==0.0.0,munch==4.0.0,natsort==8.0.2,netifaces==0.11.0,nose==1.3.7,numpy==1.21.5,oauthlib==3.2.0,olefile==0.46,packaging==21.3,Paste==3.5.0,PasteDeploy==2.1.1,PasteScript==2.0.2,pecan==1.3.3,Pillow==9.0.1,platformdirs==2.5.1,pluggy==0.13.0,ply==3.11,portend==3.0.0,prettytable==2.5.0,psutil==5.9.0,py==1.10.0,pyasn1==0.4.8,pyasn1-modules==0.2.1,Pygments==2.11.2,PyGObject==3.42.1,pyinotify==0.9.6,PyJWT==2.3.0,pymacaroons==0.13.0,PyNaCl==1.5.0,pyOpenSSL==21.0.0,pyparsing==2.4.7,pytest==6.2.5,python-apt==2.3.0+ubuntu2.1,python-dateutil==2.8.1,python-magic==0.4.24,pythran==0.10.0,pytz==2022.1,PyYAML==5.4.1,rados==2.0.0,ragweed @ file:///home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip#sha256=93390518857dde55b38ebb6bbfb1a5e0e758f094440836434fa18907f358ff00,rbd==2.0.0,registries-conf-ctl @ git+https://github.com/sebastian-philipp/registries-conf-ctl@0e9bf2dc202aa9be9b0bf8e8265aac3f2308f9db,remoto==1.2.1,repoze.lru==0.7,requests==2.25.1,requests-oauthlib==1.3.0,rgw==2.0.0,Routes==2.5.1,rsa==4.8,s3cmd==2.2.0,scikit-learn==0.23.2,scipy==1.8.0,SecretStorage==3.3.1,simplegeneric==0.8.1,simplejson==3.17.6,six==1.16.0,soupsieve==2.3.1,ssh-import-id==5.11,sympy==1.9,Tempita==0.5.2,tempora==4.1.2,threadpoolctl==3.1.0,toml==0.10.2,ubuntu-advantage-tools==27.12,ufoLib2==0.13.1,ufw==0.36.1,unattended-upgrades==0.1,unicodedata2==14.0.0,urllib3==1.26.5,virtualenv==20.13.0+ds,wadllib==1.3.6,waitress==1.4.4,wcwidth==0.2.5,webencodings==0.5.1,WebOb==1.8.6,websocket-client==1.2.3,WebTest==2.0.35,Werkzeug==2.0.2,zc.lockfile==2.0,zipp==1.0.0,zope.event==4.4,zope.interface==5.4.0 2024-03-04T17:59:28.361 INFO:teuthology.orchestra.run.smithi047.stdout:py run-test-pre: PYTHONHASHSEED='3490345998' 2024-03-04T17:59:28.361 INFO:teuthology.orchestra.run.smithi047.stdout:py run-test: commands[0] | python -m pytest -v -m 'not fails_on_rgw' 2024-03-04T17:59:28.532 INFO:teuthology.orchestra.run.smithi047.stdout:============================= test session starts ============================== 2024-03-04T17:59:28.532 INFO:teuthology.orchestra.run.smithi047.stdout:platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /home/ubuntu/cephtest/ragweed.client.0/.tox/py/bin/python 2024-03-04T17:59:28.532 INFO:teuthology.orchestra.run.smithi047.stdout:cachedir: .tox/py/.pytest_cache 2024-03-04T17:59:28.532 INFO:teuthology.orchestra.run.smithi047.stdout:rootdir: /home/ubuntu/cephtest/ragweed.client.0, configfile: pytest.ini 2024-03-04T17:59:28.693 INFO:teuthology.orchestra.run.smithi047.stdout:collecting ... collected 9 items 2024-03-04T17:59:28.693 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T17:59:40.764 INFO:teuthology.orchestra.run.smithi047.stdout:ragweed/tests/tests.py::r_test_small_obj_data::test PASSED [ 11%] 2024-03-04T17:59:54.258 INFO:teuthology.orchestra.run.smithi047.stdout:ragweed/tests/tests.py::r_test_multipart_simple::test PASSED [ 22%] 2024-03-04T18:00:05.369 INFO:teuthology.orchestra.run.smithi047.stdout:ragweed/tests/tests.py::r_test_multipart_index_versioning::test PASSED [ 33%] 2024-03-04T18:00:19.067 INFO:teuthology.orchestra.run.smithi047.stdout:ragweed/tests/tests.py::r_test_multipart_defer_complete::test PASSED [ 44%] 2024-03-04T18:00:34.006 INFO:teuthology.orchestra.run.smithi047.stdout:ragweed/tests/tests.py::r_test_multipart_defer_update_complete::test PASSED [ 55%] 2024-03-04T18:01:02.291 INFO:teuthology.orchestra.run.smithi047.stdout:ragweed/tests/tests.py::r_test_obj_storage_class::test PASSED [ 66%] 2024-03-04T18:01:28.838 INFO:teuthology.orchestra.run.smithi047.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart::test PASSED [ 77%] 2024-03-04T18:01:52.985 INFO:teuthology.orchestra.run.smithi047.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_copy::test PASSED [ 88%] 2024-03-04T18:02:40.880 INFO:teuthology.orchestra.run.smithi047.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart_copy::test PASSED [100%] 2024-03-04T18:02:40.880 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:02:40.880 INFO:teuthology.orchestra.run.smithi047.stdout:=============================== warnings summary =============================== 2024-03-04T18:02:40.881 INFO:teuthology.orchestra.run.smithi047.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2024-03-04T18:02:40.881 INFO:teuthology.orchestra.run.smithi047.stdout: /home/ubuntu/cephtest/ragweed.client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2024-03-04T18:02:40.881 INFO:teuthology.orchestra.run.smithi047.stdout: import imp 2024-03-04T18:02:40.881 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:02:40.881 INFO:teuthology.orchestra.run.smithi047.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-03-04T18:02:40.881 INFO:teuthology.orchestra.run.smithi047.stdout:=================== 9 passed, 1 warning in 192.35s (0:03:12) =================== 2024-03-04T18:02:40.938 INFO:teuthology.orchestra.run.smithi047.stdout:___________________________________ summary ____________________________________ 2024-03-04T18:02:40.938 INFO:teuthology.orchestra.run.smithi047.stdout: py: commands succeeded 2024-03-04T18:02:40.938 INFO:teuthology.orchestra.run.smithi047.stdout: congratulations :) 2024-03-04T18:02:40.973 INFO:tasks.ragweed:Cleaning up boto... 2024-03-04T18:02:40.974 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/boto.cfg 2024-03-04T18:02:40.981 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2024-03-04T18:02:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:02:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:02:41.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.105+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.105+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.105+0000 7f41a76dd980 20 realm 2024-03-04T18:02:41.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.105+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.105+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.105+0000 7f41a76dd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:02:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.105+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.105+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.105+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.109+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:02:41.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.109+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.109+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:02:41.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.109+0000 7f41a76dd980 20 searching for the correct realm 2024-03-04T18:02:41.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.113+0000 7f41a49f4640 0 -- 172.21.15.47:0/3005664865 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f419c063ce0 msgr2=0x7f419c05a920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:02:41.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.121+0000 7f41a59f6640 0 -- 172.21.15.47:0/3005664865 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x560328c1faf0 msgr2=0x560328c21fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:02:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.129+0000 7f41a76dd980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:02:41.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.129+0000 7f41a76dd980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:02:41.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.129+0000 7f41a76dd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:02:41.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.129+0000 7f41a76dd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:02:41.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.129+0000 7f41a76dd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:02:41.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.129+0000 7f41a76dd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:02:41.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.129+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.129+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.129+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 zone default found 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 4 Realm: () 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 zonegroup default 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 started sync module instance, tier type = 2024-03-04T18:02:41.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.133+0000 7f41a76dd980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:02:41.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.149+0000 7f41a76dd980 20 add_watcher() i=0 2024-03-04T18:02:41.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.149+0000 7f41a76dd980 20 add_watcher() i=1 2024-03-04T18:02:41.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.149+0000 7f41a76dd980 20 add_watcher() i=2 2024-03-04T18:02:41.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.149+0000 7f41a76dd980 20 add_watcher() i=3 2024-03-04T18:02:41.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.149+0000 7f41a76dd980 20 add_watcher() i=4 2024-03-04T18:02:41.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.149+0000 7f41a76dd980 20 add_watcher() i=5 2024-03-04T18:02:41.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.153+0000 7f41a76dd980 20 add_watcher() i=6 2024-03-04T18:02:41.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.153+0000 7f41a76dd980 20 add_watcher() i=7 2024-03-04T18:02:41.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.153+0000 7f41a76dd980 2 all 8 watchers are set, enabling cache 2024-03-04T18:02:41.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.157+0000 7f405bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:02:41.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.157+0000 7f41a76dd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:02:41.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.157+0000 7f41a76dd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:02:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f405afe5640 20 reqs_thread_entry: start 2024-03-04T18:02:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 20 init_complete bucket index max shards: 11 2024-03-04T18:02:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 20 Filter name: none 2024-03-04T18:02:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f4058fe1640 20 reqs_thread_entry: start 2024-03-04T18:02:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-03-04T18:02:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=453 2024-03-04T18:02:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x13 2024-03-04T18:02:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-03-04T18:02:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:797b67a4-b876-4d04-b122-10afd858d465.4453.4 : miss 2024-03-04T18:02:41.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:02:41.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:797b67a4-b876-4d04-b122-10afd858d465.4453.4 info.flags=0x17 2024-03-04T18:02:41.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:797b67a4-b876-4d04-b122-10afd858d465.4453.4 to cache LRU end 2024-03-04T18:02:41.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.221+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:797b67a4-b876-4d04-b122-10afd858d465.4453.4 2024-03-04T18:02:41.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.225+0000 7f41a76dd980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy 2024-03-04T18:02:41.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.225+0000 7f41a76dd980 10 cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning 2024-03-04T18:02:41.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.225+0000 7f41a76dd980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy 2024-03-04T18:02:41.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.225+0000 7f41a76dd980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete 2024-03-04T18:02:41.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.225+0000 7f41a76dd980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete 2024-03-04T18:02:41.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.225+0000 7f41a76dd980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class 2024-03-04T18:02:41.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.225+0000 7f41a76dd980 10 cls_bucket_list_unordered: got tests/r_test_multipart_simple 2024-03-04T18:02:41.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.229+0000 7f41a76dd980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart 2024-03-04T18:02:41.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.229+0000 7f41a76dd980 10 cls_bucket_list_unordered: got tests/r_test_small_obj_data 2024-03-04T18:02:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.229+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x560328cd8268 s->prefetch_data=0 2024-03-04T18:02:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.229+0000 7f41a76dd980 10 manifest: total_size = 7629 2024-03-04T18:02:41.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.229+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.9734764033580753816 2024-03-04T18:02:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.229+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x560328cd8268 s->prefetch_data=0 2024-03-04T18:02:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.229+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328cd8268 2024-03-04T18:02:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.229+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.4.1 2024-03-04T18:02:41.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.237+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x560328c62e58 s->prefetch_data=0 2024-03-04T18:02:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.237+0000 7f41a76dd980 10 manifest: total_size = 4805 2024-03-04T18:02:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.237+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.9410816691216526868 2024-03-04T18:02:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.237+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x560328c62e58 s->prefetch_data=0 2024-03-04T18:02:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.237+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c62e58 2024-03-04T18:02:41.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.237+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.4.2 2024-03-04T18:02:41.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.237+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart_copy 2024-03-04T18:02:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.241+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x560328c62e28 s->prefetch_data=0 2024-03-04T18:02:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.241+0000 7f41a76dd980 10 manifest: total_size = 15748 2024-03-04T18:02:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.241+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.17825729997900753924 2024-03-04T18:02:41.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.241+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x560328c62e28 s->prefetch_data=0 2024-03-04T18:02:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.241+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c62e28 2024-03-04T18:02:41.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.241+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.4.2 2024-03-04T18:02:41.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.241+0000 7f4077fff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_index_versioning 2024-03-04T18:02:41.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.245+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x560328c62e28 s->prefetch_data=0 2024-03-04T18:02:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.245+0000 7f41a76dd980 10 manifest: total_size = 5152 2024-03-04T18:02:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.245+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.8568927565939388429 2024-03-04T18:02:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.245+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x560328c62e28 s->prefetch_data=0 2024-03-04T18:02:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.245+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c62e28 2024-03-04T18:02:41.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.245+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.4.3 2024-03-04T18:02:41.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.249+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_copy 2024-03-04T18:02:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.253+0000 7f41a76dd980 20 get_obj_state: octx=0x560328cd8df0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x560328c62e28 s->prefetch_data=0 2024-03-04T18:02:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.253+0000 7f41a76dd980 10 manifest: total_size = 12913 2024-03-04T18:02:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.253+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.11772772492469028632 2024-03-04T18:02:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.253+0000 7f41a76dd980 20 get_obj_state: octx=0x560328cd8df0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x560328c62e28 s->prefetch_data=0 2024-03-04T18:02:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.253+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c62e28 2024-03-04T18:02:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.253+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.4.4 2024-03-04T18:02:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.257+0000 7f4077fff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_update_complete 2024-03-04T18:02:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.257+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c63c00 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x560328c62928 s->prefetch_data=0 2024-03-04T18:02:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.261+0000 7f41a76dd980 10 manifest: total_size = 7554 2024-03-04T18:02:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.261+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.14122402390598229030 2024-03-04T18:02:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.261+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c63c00 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x560328c62928 s->prefetch_data=0 2024-03-04T18:02:41.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.261+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c62928 2024-03-04T18:02:41.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.261+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.4.4 2024-03-04T18:02:41.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.261+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_complete 2024-03-04T18:02:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.265+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c5d950 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x560328c62928 s->prefetch_data=0 2024-03-04T18:02:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.265+0000 7f41a76dd980 10 manifest: total_size = 4819 2024-03-04T18:02:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.265+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.10796541002367619169 2024-03-04T18:02:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.265+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c5d950 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x560328c62928 s->prefetch_data=0 2024-03-04T18:02:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.265+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c62928 2024-03-04T18:02:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.265+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.4.5 2024-03-04T18:02:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.265+0000 7f4077fff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class 2024-03-04T18:02:41.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.269+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419807e860 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x560328c62928 s->prefetch_data=0 2024-03-04T18:02:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.269+0000 7f41a76dd980 10 manifest: total_size = 7702 2024-03-04T18:02:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.269+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.16156436607837913648 2024-03-04T18:02:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.269+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419807e860 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x560328c62928 s->prefetch_data=0 2024-03-04T18:02:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.269+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c62928 2024-03-04T18:02:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.269+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.4.7 2024-03-04T18:02:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.269+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_simple 2024-03-04T18:02:41.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.277+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419807e860 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x560328c62928 s->prefetch_data=0 2024-03-04T18:02:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.277+0000 7f41a76dd980 10 manifest: total_size = 5301 2024-03-04T18:02:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.277+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.676517803433079354 2024-03-04T18:02:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.277+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419807e860 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x560328c62928 s->prefetch_data=0 2024-03-04T18:02:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.277+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c62928 2024-03-04T18:02:41.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.277+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.4.8 2024-03-04T18:02:41.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.281+0000 7f4077fff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart 2024-03-04T18:02:41.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.285+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.285+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.289+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=tests/r_test_small_obj_data 2024-03-04T18:02:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.289+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.289+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.289+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.289+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.289+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[797b67a4-b876-4d04-b122-10afd858d465.4453.4])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.293+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2024-03-04T18:02:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.293+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.293+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=204 2024-03-04T18:02:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.293+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x11 2024-03-04T18:02:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.293+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2024-03-04T18:02:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.297+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-conf from cache 2024-03-04T18:02:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.297+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-conf, ofs0, ns] 2024-03-04T18:02:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.297+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595085 cookie 94571568697856 notifier 5928 bl.length()=162 2024-03-04T18:02:41.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.301+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:797b67a4-b876-4d04-b122-10afd858d465.4453.4 from cache 2024-03-04T18:02:41.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.301+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-conf:797b67a4-b876-4d04-b122-10afd858d465.4453.4, ofs0, ns] 2024-03-04T18:02:41.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.301+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595086 cookie 94571568697856 notifier 5928 bl.length()=219 2024-03-04T18:02:41.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.309+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-conf/797b67a4-b876-4d04-b122-10afd858d465.4453.4 : miss 2024-03-04T18:02:41.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.309+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.309+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.309+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-conf/797b67a4-b876-4d04-b122-10afd858d465.4453.4 info.flags=0x0 2024-03-04T18:02:41.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.309+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-conf/797b67a4-b876-4d04-b122-10afd858d465.4453.4 to cache LRU end 2024-03-04T18:02:41.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.313+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.11 : miss 2024-03-04T18:02:41.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.313+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.317+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=409 2024-03-04T18:02:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.317+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.11 info.flags=0x17 2024-03-04T18:02:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.317+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.11 to cache LRU end 2024-03-04T18:02:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.317+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.317+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.317+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.11 2024-03-04T18:02:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.321+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo 2024-03-04T18:02:41.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.321+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x560328c60818 s->prefetch_data=0 2024-03-04T18:02:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.321+0000 7f41a76dd980 10 manifest: total_size = 1667 2024-03-04T18:02:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.321+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.2877775117125025815 2024-03-04T18:02:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.321+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x560328c60818 s->prefetch_data=0 2024-03-04T18:02:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.321+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c60818 2024-03-04T18:02:41.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.321+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.11.10 2024-03-04T18:02:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-03-04T18:02:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=516 2024-03-04T18:02:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-03-04T18:02:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-03-04T18:02:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190724 part_ofs=516 rule->part_size=516 2024-03-04T18:02:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-03-04T18:02:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-03-04T18:02:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4191240 part_ofs=1032 rule->part_size=516 2024-03-04T18:02:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4191756 part_ofs=1548 rule->part_size=119 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.2877775117125025815 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.11__multipart_foo.2~gaCleU2Mrf7rGG3WQ95WLbIpLpDWAVP.1 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 346 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.11__multipart_foo.2~gaCleU2Mrf7rGG3WQ95WLbIpLpDWAVP.2 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 592 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.11__multipart_foo.2~gaCleU2Mrf7rGG3WQ95WLbIpLpDWAVP.3 2024-03-04T18:02:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 838 2024-03-04T18:02:41.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.11__multipart_foo.2~gaCleU2Mrf7rGG3WQ95WLbIpLpDWAVP.4 2024-03-04T18:02:41.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1084 2024-03-04T18:02:41.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.325+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.2877775117125025815 2024-03-04T18:02:41.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.329+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo 2024-03-04T18:02:41.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.329+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.329+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.333+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.333+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.333+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.333+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.333+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[797b67a4-b876-4d04-b122-10afd858d465.4453.11])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.337+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2024-03-04T18:02:41.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.337+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.337+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=235 2024-03-04T18:02:41.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.337+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x11 2024-03-04T18:02:41.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.337+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2024-03-04T18:02:41.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.337+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2024-03-04T18:02:41.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.337+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1, ofs0, ns] 2024-03-04T18:02:41.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.337+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595187 cookie 94571568742720 notifier 5928 bl.length()=191 2024-03-04T18:02:41.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.341+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.11 from cache 2024-03-04T18:02:41.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.341+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.11, ofs0, ns] 2024-03-04T18:02:41.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.341+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595092 cookie 94571568746912 notifier 5928 bl.length()=249 2024-03-04T18:02:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.349+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/797b67a4-b876-4d04-b122-10afd858d465.4453.11 : miss 2024-03-04T18:02:41.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.349+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.349+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.349+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/797b67a4-b876-4d04-b122-10afd858d465.4453.11 info.flags=0x0 2024-03-04T18:02:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.349+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/797b67a4-b876-4d04-b122-10afd858d465.4453.11 to cache LRU end 2024-03-04T18:02:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.353+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.12 : miss 2024-03-04T18:02:41.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.353+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.357+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=416 2024-03-04T18:02:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.357+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.12 info.flags=0x17 2024-03-04T18:02:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.357+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.12 to cache LRU end 2024-03-04T18:02:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.357+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.357+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.357+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.12 2024-03-04T18:02:41.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.361+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo 2024-03-04T18:02:41.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.361+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c009400 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x560328c5ddd8 s->prefetch_data=0 2024-03-04T18:02:41.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.361+0000 7f41a76dd980 10 manifest: total_size = 26080 2024-03-04T18:02:41.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.361+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.13102855747637122916 2024-03-04T18:02:41.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.361+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c009400 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x560328c5ddd8 s->prefetch_data=0 2024-03-04T18:02:41.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.361+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5ddd8 2024-03-04T18:02:41.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.361+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.12.10 2024-03-04T18:02:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=7673 rules.size()=4 2024-03-04T18:02:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=7673 2024-03-04T18:02:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=7673 stripe_ofs=7673 part_ofs=7673 rule->part_size=7668 2024-03-04T18:02:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=7668 rules.size()=4 2024-03-04T18:02:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4197881 part_ofs=7673 rule->part_size=7668 2024-03-04T18:02:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=15341 stripe_ofs=15341 part_ofs=15341 rule->part_size=7672 2024-03-04T18:02:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=4 2024-03-04T18:02:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4205549 part_ofs=15341 rule->part_size=7672 2024-03-04T18:02:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=23013 stripe_ofs=23013 part_ofs=23013 rule->part_size=3067 2024-03-04T18:02:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=4 2024-03-04T18:02:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4213221 part_ofs=23013 rule->part_size=3067 2024-03-04T18:02:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=26080 stripe_ofs=26080 part_ofs=26080 rule->part_size=3067 2024-03-04T18:02:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.13102855747637122916 2024-03-04T18:02:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.12__multipart_foo.2~lp4wl5liFv-J_rfgX_9OgtQRAx3ofnK.1 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.12__multipart_foo.2~lp4wl5liFv-J_rfgX_9OgtQRAx3ofnK.2 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.12__multipart_foo.2~lp4wl5liFv-J_rfgX_9OgtQRAx3ofnK.3 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.12__multipart_foo.2~lp4wl5liFv-J_rfgX_9OgtQRAx3ofnK.4 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.13102855747637122916 2024-03-04T18:02:41.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.369+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo 2024-03-04T18:02:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.373+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.373+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.373+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.373+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.373+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.373+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.373+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[797b67a4-b876-4d04-b122-10afd858d465.4453.12])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.377+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2024-03-04T18:02:41.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.377+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.377+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=242 2024-03-04T18:02:41.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.377+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x11 2024-03-04T18:02:41.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.377+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2024-03-04T18:02:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.381+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2024-03-04T18:02:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.381+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1, ofs0, ns] 2024-03-04T18:02:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.381+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595093 cookie 94571568746912 notifier 5928 bl.length()=198 2024-03-04T18:02:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.385+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.12 from cache 2024-03-04T18:02:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.385+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:797b67a4-b876-4d04-b122-10afd858d465.4453.12, ofs0, ns] 2024-03-04T18:02:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.385+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595096 cookie 94571568760400 notifier 5928 bl.length()=256 2024-03-04T18:02:41.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.397+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/797b67a4-b876-4d04-b122-10afd858d465.4453.12 : miss 2024-03-04T18:02:41.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.397+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.397+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.397+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/797b67a4-b876-4d04-b122-10afd858d465.4453.12 info.flags=0x0 2024-03-04T18:02:41.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.397+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/797b67a4-b876-4d04-b122-10afd858d465.4453.12 to cache LRU end 2024-03-04T18:02:41.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.401+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:797b67a4-b876-4d04-b122-10afd858d465.4453.10 : miss 2024-03-04T18:02:41.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.401+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.401+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=411 2024-03-04T18:02:41.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.401+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:797b67a4-b876-4d04-b122-10afd858d465.4453.10 info.flags=0x17 2024-03-04T18:02:41.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.401+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:797b67a4-b876-4d04-b122-10afd858d465.4453.10 to cache LRU end 2024-03-04T18:02:41.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.401+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.401+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.401+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:797b67a4-b876-4d04-b122-10afd858d465.4453.10 2024-03-04T18:02:41.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.409+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo[4O6tzs9M3YOIWuCaQM.kASS90tcOx6.] 2024-03-04T18:02:41.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.409+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.10.10 2024-03-04T18:02:41.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.409+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c59f10 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x560328c5ddd8 s->prefetch_data=0 2024-03-04T18:02:41.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.409+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to bdffcofavfb13wyroy48ymf87e35ewrb 2024-03-04T18:02:41.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.409+0000 7f41a76dd980 20 get_obj_state_impl: setting s->olh_tag to q8y77toydhszx44jx7ouj2rgfuib7onb 2024-03-04T18:02:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.409+0000 7f41a76dd980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=q8y77toydhszx44jx7ouj2rgfuib7onb 2024-03-04T18:02:41.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.413+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.10.10 2024-03-04T18:02:41.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.413+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.10.10 2024-03-04T18:02:41.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.417+0000 7f41a76dd980 20 olh_log_entry: epoch=3 op=2 key=foo[] 2024-03-04T18:02:41.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.417+0000 7f41a76dd980 20 olh_log_entry: epoch=3 op=3 key=foo[4O6tzs9M3YOIWuCaQM.kASS90tcOx6.] 2024-03-04T18:02:41.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.417+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c59f10 obj=test-client.0-r-test-multipart-index-versioning.1:_:4O6tzs9M3YOIWuCaQM.kASS90tcOx6._foo state=0x560328c65c08 s->prefetch_data=0 2024-03-04T18:02:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.417+0000 7f41a76dd980 10 manifest: total_size = 74198 2024-03-04T18:02:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.417+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.9789271116981981130 2024-03-04T18:02:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.417+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c59f10 obj=test-client.0-r-test-multipart-index-versioning.1:_:4O6tzs9M3YOIWuCaQM.kASS90tcOx6._foo state=0x560328c65c08 s->prefetch_data=0 2024-03-04T18:02:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.417+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c65c08 2024-03-04T18:02:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.417+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.10.10 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=20613 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4210821 part_ofs=20613 rule->part_size=20613 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4231434 part_ofs=41226 rule->part_size=20613 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2024-03-04T18:02:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4252047 part_ofs=61839 rule->part_size=12359 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.9789271116981981130 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.10__multipart_foo.2~o-nCrA4GpdQ2cUm9wefoFtvV4H8xKhL.1 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 346 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.10__multipart_foo.2~o-nCrA4GpdQ2cUm9wefoFtvV4H8xKhL.2 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 592 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.10__multipart_foo.2~o-nCrA4GpdQ2cUm9wefoFtvV4H8xKhL.3 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 838 2024-03-04T18:02:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.10__multipart_foo.2~o-nCrA4GpdQ2cUm9wefoFtvV4H8xKhL.4 2024-03-04T18:02:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1084 2024-03-04T18:02:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.421+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.9789271116981981130 2024-03-04T18:02:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.425+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo[4O6tzs9M3YOIWuCaQM.kASS90tcOx6.] 2024-03-04T18:02:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.429+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c59f10 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x560328c5ddd8 s->prefetch_data=0 2024-03-04T18:02:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.429+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.10.10 2024-03-04T18:02:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.433+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.10.10 2024-03-04T18:02:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.433+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.433+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.433+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.433+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.433+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.433+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.433+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[797b67a4-b876-4d04-b122-10afd858d465.4453.10])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.441+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2024-03-04T18:02:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.441+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.441+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=237 2024-03-04T18:02:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.441+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x11 2024-03-04T18:02:41.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.441+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2024-03-04T18:02:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.445+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2024-03-04T18:02:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.445+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1, ofs0, ns] 2024-03-04T18:02:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.445+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595094 cookie 94571568746912 notifier 5928 bl.length()=193 2024-03-04T18:02:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.449+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:797b67a4-b876-4d04-b122-10afd858d465.4453.10 from cache 2024-03-04T18:02:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.449+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:797b67a4-b876-4d04-b122-10afd858d465.4453.10, ofs0, ns] 2024-03-04T18:02:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.449+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595097 cookie 94571568754160 notifier 5928 bl.length()=251 2024-03-04T18:02:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.457+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/797b67a4-b876-4d04-b122-10afd858d465.4453.10 : miss 2024-03-04T18:02:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.457+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.457+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.457+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/797b67a4-b876-4d04-b122-10afd858d465.4453.10 info.flags=0x0 2024-03-04T18:02:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.457+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/797b67a4-b876-4d04-b122-10afd858d465.4453.10 to cache LRU end 2024-03-04T18:02:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.461+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:797b67a4-b876-4d04-b122-10afd858d465.4453.9 : miss 2024-03-04T18:02:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.461+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.465+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=399 2024-03-04T18:02:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.465+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:797b67a4-b876-4d04-b122-10afd858d465.4453.9 info.flags=0x17 2024-03-04T18:02:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.465+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:797b67a4-b876-4d04-b122-10afd858d465.4453.9 to cache LRU end 2024-03-04T18:02:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.465+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.465+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.465+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:797b67a4-b876-4d04-b122-10afd858d465.4453.9 2024-03-04T18:02:41.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.473+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo 2024-03-04T18:02:41.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.473+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c672b0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x560328c71c68 s->prefetch_data=0 2024-03-04T18:02:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.473+0000 7f41a76dd980 10 manifest: total_size = 1667 2024-03-04T18:02:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.473+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.9679328132611048601 2024-03-04T18:02:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.473+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c672b0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x560328c71c68 s->prefetch_data=0 2024-03-04T18:02:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.473+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c71c68 2024-03-04T18:02:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.473+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.9.10 2024-03-04T18:02:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-03-04T18:02:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=516 2024-03-04T18:02:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-03-04T18:02:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-03-04T18:02:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190724 part_ofs=516 rule->part_size=516 2024-03-04T18:02:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-03-04T18:02:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-03-04T18:02:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4191240 part_ofs=1032 rule->part_size=516 2024-03-04T18:02:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4191756 part_ofs=1548 rule->part_size=119 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.9679328132611048601 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.9__multipart_foo.2~2KqC2PNgWr40jRqZVhF8UQumGSCoeDH.1 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 344 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.9__multipart_foo.2~2KqC2PNgWr40jRqZVhF8UQumGSCoeDH.2 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 588 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.9__multipart_foo.2~2KqC2PNgWr40jRqZVhF8UQumGSCoeDH.3 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 832 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.9__multipart_foo.2~2KqC2PNgWr40jRqZVhF8UQumGSCoeDH.4 2024-03-04T18:02:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1076 2024-03-04T18:02:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.477+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.9679328132611048601 2024-03-04T18:02:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.481+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo 2024-03-04T18:02:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.481+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.481+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.485+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.485+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.485+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.485+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.485+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[797b67a4-b876-4d04-b122-10afd858d465.4453.9])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.489+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2024-03-04T18:02:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.489+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.489+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=225 2024-03-04T18:02:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.489+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x11 2024-03-04T18:02:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.489+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2024-03-04T18:02:41.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.493+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2024-03-04T18:02:41.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.493+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-simple.1, ofs0, ns] 2024-03-04T18:02:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.493+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595188 cookie 94571568749856 notifier 5928 bl.length()=183 2024-03-04T18:02:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.497+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:797b67a4-b876-4d04-b122-10afd858d465.4453.9 from cache 2024-03-04T18:02:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.497+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:797b67a4-b876-4d04-b122-10afd858d465.4453.9, ofs0, ns] 2024-03-04T18:02:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.497+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595098 cookie 94571568757760 notifier 5928 bl.length()=240 2024-03-04T18:02:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.505+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/797b67a4-b876-4d04-b122-10afd858d465.4453.9 : miss 2024-03-04T18:02:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.505+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.505+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.505+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/797b67a4-b876-4d04-b122-10afd858d465.4453.9 info.flags=0x0 2024-03-04T18:02:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.505+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/797b67a4-b876-4d04-b122-10afd858d465.4453.9 to cache LRU end 2024-03-04T18:02:41.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.509+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.15 : miss 2024-03-04T18:02:41.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.509+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.513+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=407 2024-03-04T18:02:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.513+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.15 info.flags=0x17 2024-03-04T18:02:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.513+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.15 to cache LRU end 2024-03-04T18:02:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.513+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.513+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.513+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.15 2024-03-04T18:02:41.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.513+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-03-04T18:02:41.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.513+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-03-04T18:02:41.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.517+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-03-04T18:02:41.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.517+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-03-04T18:02:41.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.517+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-03-04T18:02:41.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.517+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-03-04T18:02:41.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.517+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-03-04T18:02:41.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.517+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-03-04T18:02:41.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.517+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-03-04T18:02:41.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.521+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-03-04T18:02:41.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.521+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-03-04T18:02:41.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.521+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-03-04T18:02:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.521+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c5ebc0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.521+0000 7f41a76dd980 10 manifest: total_size = 22651 2024-03-04T18:02:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.521+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _gPqqm_l0B9XgGBwuy_a2xGcjviGZ_Uz 2024-03-04T18:02:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.521+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c5ebc0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.521+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.521+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.0 2024-03-04T18:02:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.525+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.525+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=22651 stripe_ofs=22651 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.525+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _gPqqm_l0B9XgGBwuy_a2xGcjviGZ_Uz 2024-03-04T18:02:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.525+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.525+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.4aEqhPJAnvyypGuxfbWx98cUiJpQpOf_0 2024-03-04T18:02:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.525+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-03-04T18:02:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.525+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.525+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: _gPqqm_l0B9XgGBwuy_a2xGcjviGZ_Uz 2024-03-04T18:02:41.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.529+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-03-04T18:02:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.529+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c5ebc0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.533+0000 7f41a76dd980 10 manifest: total_size = 5242880 2024-03-04T18:02:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.533+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _YOv8w7R6AywxdAI1h_wT9nRvpK0DSgu 2024-03-04T18:02:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.533+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c5ebc0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.533+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.533+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.2 2024-03-04T18:02:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.537+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.537+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.537+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _YOv8w7R6AywxdAI1h_wT9nRvpK0DSgu 2024-03-04T18:02:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.537+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.537+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.-YGKFu-omL6wq4yRHliGfuEKMIKSgNZ_1 2024-03-04T18:02:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.537+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-03-04T18:02:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.537+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.537+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: _YOv8w7R6AywxdAI1h_wT9nRvpK0DSgu 2024-03-04T18:02:41.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.541+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c5ebc0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.541+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-03-04T18:02:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.541+0000 7f41a76dd980 10 manifest: total_size = 5242880 2024-03-04T18:02:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.541+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.14005680552609425226 2024-03-04T18:02:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.541+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c5ebc0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.541+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.541+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.3 2024-03-04T18:02:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.14005680552609425226 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_0 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 338 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_1 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 575 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.545+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.14005680552609425226 2024-03-04T18:02:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.549+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-03-04T18:02:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.549+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.549+0000 7f41a76dd980 10 manifest: total_size = 5242880 2024-03-04T18:02:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.549+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _tCs2AFYIcKe5YYCSG2puNk8SrzQDHSy 2024-03-04T18:02:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.549+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.549+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.549+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.5 2024-03-04T18:02:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _tCs2AFYIcKe5YYCSG2puNk8SrzQDHSy 2024-03-04T18:02:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.iIvWexlfPpdG5UdXzEa5hFa0enCDsVi_0 2024-03-04T18:02:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-03-04T18:02:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.iIvWexlfPpdG5UdXzEa5hFa0enCDsVi_1 2024-03-04T18:02:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-03-04T18:02:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: _tCs2AFYIcKe5YYCSG2puNk8SrzQDHSy 2024-03-04T18:02:41.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-03-04T18:02:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.557+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00bfda0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.561+0000 7f41a76dd980 10 manifest: total_size = 5242880 2024-03-04T18:02:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.561+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _LKTUjfqrj98u7_d8vMM9YQuJTOv_sRt 2024-03-04T18:02:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.561+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00bfda0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.561+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.561+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.5 2024-03-04T18:02:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _LKTUjfqrj98u7_d8vMM9YQuJTOv_sRt 2024-03-04T18:02:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.CfwNpRssXVb2FlZB9zdy37552CpbuuJ_0 2024-03-04T18:02:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-03-04T18:02:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.CfwNpRssXVb2FlZB9zdy37552CpbuuJ_1 2024-03-04T18:02:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-03-04T18:02:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _LKTUjfqrj98u7_d8vMM9YQuJTOv_sRt 2024-03-04T18:02:41.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.565+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-03-04T18:02:41.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.569+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.569+0000 7f41a76dd980 10 manifest: total_size = 22651 2024-03-04T18:02:41.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.569+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.2489751092182844222 2024-03-04T18:02:41.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.569+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.569+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.569+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.5 2024-03-04T18:02:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.573+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.573+0000 7f41a76dd980 10 manifest: total_size = 5242880 2024-03-04T18:02:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.573+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _Q2s0T_CZ-BbrSJuTZ5Fip8uP_Ppd4Xd 2024-03-04T18:02:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.573+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.573+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.573+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.7 2024-03-04T18:02:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.577+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-03-04T18:02:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.581+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.581+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.581+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _Q2s0T_CZ-BbrSJuTZ5Fip8uP_Ppd4Xd 2024-03-04T18:02:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.581+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.581+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.WoKRM9-ZC5eihC3R41EczHbaS6OUOQm_1 2024-03-04T18:02:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.581+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-03-04T18:02:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.581+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.581+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: _Q2s0T_CZ-BbrSJuTZ5Fip8uP_Ppd4Xd 2024-03-04T18:02:41.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.585+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-03-04T18:02:41.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.585+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c631c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.585+0000 7f41a76dd980 10 manifest: total_size = 5242880 2024-03-04T18:02:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.585+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.6381786571596053396 2024-03-04T18:02:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.585+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c631c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.585+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.585+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.7 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.6381786571596053396 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_0 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 339 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_1 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 578 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.589+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.6381786571596053396 2024-03-04T18:02:41.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.593+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-03-04T18:02:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.593+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.593+0000 7f41a76dd980 10 manifest: total_size = 22651 2024-03-04T18:02:41.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.593+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _ol-F9MkLJs-BanyVvNRe_v3ZzW1egxf 2024-03-04T18:02:41.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.593+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.593+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.593+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.7 2024-03-04T18:02:41.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.601+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2f50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.601+0000 7f41a76dd980 10 manifest: total_size = 5242880 2024-03-04T18:02:41.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.601+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.601+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.601+0000 7f41a76dd980 10 generate_fake_tag new tag=797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_1_06360b9bc047dd7174c6c53cb544f79b 2024-03-04T18:02:41.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.601+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_1_06360b9bc047dd7174c6c53cb544f79b 2024-03-04T18:02:41.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.601+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2f50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.601+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.601+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.8 2024-03-04T18:02:41.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_1_06360b9bc047dd7174c6c53cb544f79b 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_0 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 398 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_1 2024-03-04T18:02:41.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 637 2024-03-04T18:02:41.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.605+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_1_06360b9bc047dd7174c6c53cb544f79b 2024-03-04T18:02:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.609+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-03-04T18:02:41.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.609+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2f50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.613+0000 7f41a76dd980 10 manifest: total_size = 22651 2024-03-04T18:02:41.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.613+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _6iG3oCpQd_gBS6npn802tsXSuTX9Rdq 2024-03-04T18:02:41.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.613+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2f50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.613+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.613+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.8 2024-03-04T18:02:41.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.617+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.617+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=22651 stripe_ofs=22651 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.617+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _6iG3oCpQd_gBS6npn802tsXSuTX9Rdq 2024-03-04T18:02:41.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.617+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.617+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.dRmV9MS-L3--wEXNaN0P5Tn4ekbEUxC_0 2024-03-04T18:02:41.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.617+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-03-04T18:02:41.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.617+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.617+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _6iG3oCpQd_gBS6npn802tsXSuTX9Rdq 2024-03-04T18:02:41.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-03-04T18:02:41.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f41a76dd980 10 manifest: total_size = 5242880 2024-03-04T18:02:41.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f41a76dd980 10 generate_fake_tag new tag=797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_1_eddb828882091a3d9c8cf3afdac2b7cf 2024-03-04T18:02:41.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_1_eddb828882091a3d9c8cf3afdac2b7cf 2024-03-04T18:02:41.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x560328c73428 s->prefetch_data=0 2024-03-04T18:02:41.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c73428 2024-03-04T18:02:41.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.621+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.15.9 2024-03-04T18:02:41.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_1_eddb828882091a3d9c8cf3afdac2b7cf 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_0 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 396 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_1 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 633 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_1_eddb828882091a3d9c8cf3afdac2b7cf 2024-03-04T18:02:41.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.629+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-03-04T18:02:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.633+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.633+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.633+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.633+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.633+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.633+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.633+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[797b67a4-b876-4d04-b122-10afd858d465.4453.15])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.637+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2024-03-04T18:02:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.637+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.637+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=233 2024-03-04T18:02:41.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.637+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x11 2024-03-04T18:02:41.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.637+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2024-03-04T18:02:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.641+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2024-03-04T18:02:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.641+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1, ofs0, ns] 2024-03-04T18:02:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.641+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595095 cookie 94571568746912 notifier 5928 bl.length()=189 2024-03-04T18:02:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.645+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.15 from cache 2024-03-04T18:02:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.645+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.15, ofs0, ns] 2024-03-04T18:02:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.645+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595099 cookie 94571568754160 notifier 5928 bl.length()=247 2024-03-04T18:02:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.657+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/797b67a4-b876-4d04-b122-10afd858d465.4453.15 : miss 2024-03-04T18:02:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.657+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.657+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.657+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/797b67a4-b876-4d04-b122-10afd858d465.4453.15 info.flags=0x0 2024-03-04T18:02:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.657+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/797b67a4-b876-4d04-b122-10afd858d465.4453.15 to cache LRU end 2024-03-04T18:02:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.661+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.16 : miss 2024-03-04T18:02:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.661+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.661+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=417 2024-03-04T18:02:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.661+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.16 info.flags=0x17 2024-03-04T18:02:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.661+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.16 to cache LRU end 2024-03-04T18:02:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.661+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.661+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.661+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.16 2024-03-04T18:02:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.661+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-03-04T18:02:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.665+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-03-04T18:02:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.665+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-03-04T18:02:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.669+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-03-04T18:02:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.669+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-03-04T18:02:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.669+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-03-04T18:02:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.669+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-03-04T18:02:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.669+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-03-04T18:02:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.669+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-03-04T18:02:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.669+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-03-04T18:02:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.669+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-03-04T18:02:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.673+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-03-04T18:02:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.673+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c0222e0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.673+0000 7f41a76dd980 10 manifest: total_size = 45360 2024-03-04T18:02:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.673+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _wTykr_axHiR9fiAMD-UX47L12hTOKpT 2024-03-04T18:02:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.673+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c0222e0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.673+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.673+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.0 2024-03-04T18:02:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.677+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.677+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.677+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _wTykr_axHiR9fiAMD-UX47L12hTOKpT 2024-03-04T18:02:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.677+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.677+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.YZVxQvz917Gfc6is0xT5L3aLBMaH7HO_0 2024-03-04T18:02:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.677+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-03-04T18:02:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.677+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.677+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: _wTykr_axHiR9fiAMD-UX47L12hTOKpT 2024-03-04T18:02:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.681+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2c50 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.681+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-03-04T18:02:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.681+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.681+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _qeyxFPzw0wZ9pbj4NJjctv4nZFghtFm 2024-03-04T18:02:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.681+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2c50 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.681+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.681+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.2 2024-03-04T18:02:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _qeyxFPzw0wZ9pbj4NJjctv4nZFghtFm 2024-03-04T18:02:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.4dH8ROPc6W-kfP0Yew_niVHlRkLpd5K_1 2024-03-04T18:02:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-03-04T18:02:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.4dH8ROPc6W-kfP0Yew_niVHlRkLpd5K_2 2024-03-04T18:02:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-03-04T18:02:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.685+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: _qeyxFPzw0wZ9pbj4NJjctv4nZFghtFm 2024-03-04T18:02:41.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.689+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-03-04T18:02:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.689+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.693+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.693+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.18141961772719482821 2024-03-04T18:02:41.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.693+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.693+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.693+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.3 2024-03-04T18:02:41.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.18141961772719482821 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 368 2024-03-04T18:02:41.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1_1 2024-03-04T18:02:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 633 2024-03-04T18:02:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.2 2024-03-04T18:02:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 900 2024-03-04T18:02:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.2_1 2024-03-04T18:02:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1165 2024-03-04T18:02:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.3 2024-03-04T18:02:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1432 2024-03-04T18:02:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.18141961772719482821 2024-03-04T18:02:41.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.697+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-03-04T18:02:41.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.701+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419805bf30 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.701+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.701+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _Q3xcVOoqdG5qsL8FnCzAXR5iIbzLqwN 2024-03-04T18:02:41.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.701+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419805bf30 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.701+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.701+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.5 2024-03-04T18:02:41.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _Q3xcVOoqdG5qsL8FnCzAXR5iIbzLqwN 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zegKftU_JlKUfv_KCOPSMH0qaVAJIFu_0 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zegKftU_JlKUfv_KCOPSMH0qaVAJIFu_1 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zegKftU_JlKUfv_KCOPSMH0qaVAJIFu_2 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 788 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.705+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: _Q3xcVOoqdG5qsL8FnCzAXR5iIbzLqwN 2024-03-04T18:02:41.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.709+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-03-04T18:02:41.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.709+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.709+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.709+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _w2BG5vXwycBV2zM3g_FT_z5vCB_k-j5 2024-03-04T18:02:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.709+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.709+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.709+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.5 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _w2BG5vXwycBV2zM3g_FT_z5vCB_k-j5 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zb3gqOwuD2dYLT7cpKJceLnc4DtkVGG_0 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zb3gqOwuD2dYLT7cpKJceLnc4DtkVGG_1 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zb3gqOwuD2dYLT7cpKJceLnc4DtkVGG_2 2024-03-04T18:02:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 782 2024-03-04T18:02:41.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.717+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: _w2BG5vXwycBV2zM3g_FT_z5vCB_k-j5 2024-03-04T18:02:41.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.721+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-03-04T18:02:41.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.721+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c0222e0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.721+0000 7f41a76dd980 10 manifest: total_size = 45360 2024-03-04T18:02:41.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.721+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.1739080031530623205 2024-03-04T18:02:41.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.721+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c0222e0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.721+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.721+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.5 2024-03-04T18:02:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-03-04T18:02:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=20613 2024-03-04T18:02:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2024-03-04T18:02:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-03-04T18:02:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4210821 part_ofs=20613 rule->part_size=20613 2024-03-04T18:02:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2024-03-04T18:02:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2024-03-04T18:02:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4231434 part_ofs=41226 rule->part_size=4134 2024-03-04T18:02:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.1739080031530623205 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.1 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 364 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.2 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 628 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.3 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 892 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.1739080031530623205 2024-03-04T18:02:41.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.729+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-03-04T18:02:41.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.733+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2660 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.733+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.733+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _avR_g3LXlAmO90aYmxtiTVZQ-IeS6Kj 2024-03-04T18:02:41.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.733+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2660 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.733+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.733+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.7 2024-03-04T18:02:41.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _avR_g3LXlAmO90aYmxtiTVZQ-IeS6Kj 2024-03-04T18:02:41.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.smgSYQLtElJOeW3MpE-fSrRAQobPkDN_1 2024-03-04T18:02:41.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-03-04T18:02:41.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.smgSYQLtElJOeW3MpE-fSrRAQobPkDN_2 2024-03-04T18:02:41.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-03-04T18:02:41.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.737+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: _avR_g3LXlAmO90aYmxtiTVZQ-IeS6Kj 2024-03-04T18:02:41.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.741+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419805bf30 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.741+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-03-04T18:02:41.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.741+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.741+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.6702748022887742919 2024-03-04T18:02:41.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.741+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419805bf30 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.741+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.741+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.7 2024-03-04T18:02:41.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.6702748022887742919 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 373 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1_1 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 644 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.2 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 917 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.2_1 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1188 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.3 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1461 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.6702748022887742919 2024-03-04T18:02:41.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c67030 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.749+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-03-04T18:02:41.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.753+0000 7f41a76dd980 10 manifest: total_size = 45360 2024-03-04T18:02:41.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.753+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-03-04T18:02:41.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.753+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=20613 2024-03-04T18:02:41.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.753+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2024-03-04T18:02:41.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.753+0000 7f41a76dd980 10 generate_fake_tag new tag=797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.2_ee25a3e7762809002588a4ae7bc98789 2024-03-04T18:02:41.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.753+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.2_ee25a3e7762809002588a4ae7bc98789 2024-03-04T18:02:41.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.753+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c67030 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.753+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.753+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.7 2024-03-04T18:02:41.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-03-04T18:02:41.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=20613 2024-03-04T18:02:41.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2024-03-04T18:02:41.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4210821 part_ofs=20613 rule->part_size=20613 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4231434 part_ofs=41226 rule->part_size=4134 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.2_ee25a3e7762809002588a4ae7bc98789 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.1 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.2 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 704 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.3 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 968 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.757+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.2_ee25a3e7762809002588a4ae7bc98789 2024-03-04T18:02:41.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c35d10 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-03-04T18:02:41.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 10 generate_fake_tag new tag=797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1_1_0a4854d21ffe13c6750b99a9f117ed7a 2024-03-04T18:02:41.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1_1_0a4854d21ffe13c6750b99a9f117ed7a 2024-03-04T18:02:41.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c35d10 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.761+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.8 2024-03-04T18:02:41.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1_1_0a4854d21ffe13c6750b99a9f117ed7a 2024-03-04T18:02:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 448 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1_1 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 719 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.2 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 992 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.2_1 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1263 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.3 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1536 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.765+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1_1_0a4854d21ffe13c6750b99a9f117ed7a 2024-03-04T18:02:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.769+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-03-04T18:02:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.769+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c6ced0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.769+0000 7f41a76dd980 10 manifest: total_size = 45360 2024-03-04T18:02:41.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.769+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to _fug73i_hhm41yXv34zkblxchvQXjvob 2024-03-04T18:02:41.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.769+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c6ced0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.769+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.769+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.8 2024-03-04T18:02:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.777+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.777+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.777+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: _fug73i_hhm41yXv34zkblxchvQXjvob 2024-03-04T18:02:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.777+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.777+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.ElTGv4y1Va1jJ_ngWtnkEdhNs_s3Xw5_0 2024-03-04T18:02:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.777+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-03-04T18:02:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.777+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.777+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _fug73i_hhm41yXv34zkblxchvQXjvob 2024-03-04T18:02:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.777+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c0222e0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-03-04T18:02:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f41a76dd980 10 generate_fake_tag new tag=797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1_1_bb7763219b770929e6480b31170d9ca1 2024-03-04T18:02:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1_1_bb7763219b770929e6480b31170d9ca1 2024-03-04T18:02:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c0222e0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x560328c5c6b8 s->prefetch_data=0 2024-03-04T18:02:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c6b8 2024-03-04T18:02:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.781+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.16.9 2024-03-04T18:02:41.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1_1_bb7763219b770929e6480b31170d9ca1 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1_1 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 705 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.2 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 972 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.2_1 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1237 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.3 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1504 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.785+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1_1_bb7763219b770929e6480b31170d9ca1 2024-03-04T18:02:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.789+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-03-04T18:02:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.789+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.789+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.789+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.789+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.789+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.789+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.789+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[797b67a4-b876-4d04-b122-10afd858d465.4453.16])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.793+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2024-03-04T18:02:41.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.793+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.793+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=243 2024-03-04T18:02:41.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.793+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x11 2024-03-04T18:02:41.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.793+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2024-03-04T18:02:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.797+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2024-03-04T18:02:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.797+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1, ofs0, ns] 2024-03-04T18:02:41.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.797+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595090 cookie 94571568700064 notifier 5928 bl.length()=199 2024-03-04T18:02:41.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.801+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.16 from cache 2024-03-04T18:02:41.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.801+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:797b67a4-b876-4d04-b122-10afd858d465.4453.16, ofs0, ns] 2024-03-04T18:02:41.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.801+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595087 cookie 94571568697856 notifier 5928 bl.length()=257 2024-03-04T18:02:41.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.809+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/797b67a4-b876-4d04-b122-10afd858d465.4453.16 : miss 2024-03-04T18:02:41.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.809+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.809+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.809+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/797b67a4-b876-4d04-b122-10afd858d465.4453.16 info.flags=0x0 2024-03-04T18:02:41.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.809+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/797b67a4-b876-4d04-b122-10afd858d465.4453.16 to cache LRU end 2024-03-04T18:02:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.813+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:797b67a4-b876-4d04-b122-10afd858d465.4453.14 : miss 2024-03-04T18:02:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.813+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.813+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=412 2024-03-04T18:02:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.813+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:797b67a4-b876-4d04-b122-10afd858d465.4453.14 info.flags=0x17 2024-03-04T18:02:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.813+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:797b67a4-b876-4d04-b122-10afd858d465.4453.14 to cache LRU end 2024-03-04T18:02:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.813+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.813+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.813+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:797b67a4-b876-4d04-b122-10afd858d465.4453.14 2024-03-04T18:02:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.817+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-03-04T18:02:41.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.817+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-03-04T18:02:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.821+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-03-04T18:02:41.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.821+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c73dc0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x560328c5c468 s->prefetch_data=0 2024-03-04T18:02:41.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.821+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.821+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.13233331196442993742 2024-03-04T18:02:41.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.821+0000 7f41a76dd980 20 get_obj_state: octx=0x560328c73dc0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x560328c5c468 s->prefetch_data=0 2024-03-04T18:02:41.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.821+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c468 2024-03-04T18:02:41.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.825+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.14.3 2024-03-04T18:02:41.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.13233331196442993742 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.1 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 368 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__shadow_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.1_1 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 633 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.2 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 900 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__shadow_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.2_1 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1165 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.3 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1432 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.13233331196442993742 2024-03-04T18:02:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-03-04T18:02:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.829+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x560328c5c468 s->prefetch_data=0 2024-03-04T18:02:41.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.833+0000 7f41a76dd980 10 manifest: total_size = 1087 2024-03-04T18:02:41.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.833+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.6897579346510462014 2024-03-04T18:02:41.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.833+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c089c50 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x560328c5c468 s->prefetch_data=0 2024-03-04T18:02:41.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.833+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c468 2024-03-04T18:02:41.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.833+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.14.5 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190208 part_ofs=0 rule->part_size=516 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4190724 part_ofs=516 rule->part_size=516 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=55 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=55 rules.size()=2 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4191240 part_ofs=1032 rule->part_size=55 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=1087 rule->part_size=55 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.6897579346510462014 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-STANDARD.2~oNqtygdxtILu9rgeuNSwdKSplrTRmFw.1 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 364 2024-03-04T18:02:41.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-STANDARD.2~oNqtygdxtILu9rgeuNSwdKSplrTRmFw.2 2024-03-04T18:02:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 628 2024-03-04T18:02:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-STANDARD.2~oNqtygdxtILu9rgeuNSwdKSplrTRmFw.3 2024-03-04T18:02:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 892 2024-03-04T18:02:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.6897579346510462014 2024-03-04T18:02:41.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-03-04T18:02:41.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.837+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419805bf30 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x560328c5c468 s->prefetch_data=0 2024-03-04T18:02:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.841+0000 7f41a76dd980 10 manifest: total_size = 11534336 2024-03-04T18:02:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.841+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.16872373066293482154 2024-03-04T18:02:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.841+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419805bf30 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x560328c5c468 s->prefetch_data=0 2024-03-04T18:02:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.841+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c5c468 2024-03-04T18:02:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.841+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.14.7 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.16872373066293482154 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.1 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 374 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__shadow_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.1_1 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 645 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.2 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 918 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__shadow_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.2_1 2024-03-04T18:02:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1189 2024-03-04T18:02:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.3 2024-03-04T18:02:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1462 2024-03-04T18:02:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.845+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.16872373066293482154 2024-03-04T18:02:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.849+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-03-04T18:02:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.849+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.849+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.849+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.849+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.849+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.849+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.849+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[797b67a4-b876-4d04-b122-10afd858d465.4453.14])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.853+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2024-03-04T18:02:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.853+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.853+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=238 2024-03-04T18:02:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.853+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x11 2024-03-04T18:02:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.853+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2024-03-04T18:02:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.857+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2024-03-04T18:02:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.857+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1, ofs0, ns] 2024-03-04T18:02:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.857+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595100 cookie 94571568754160 notifier 5928 bl.length()=194 2024-03-04T18:02:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.861+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:797b67a4-b876-4d04-b122-10afd858d465.4453.14 from cache 2024-03-04T18:02:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.861+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:797b67a4-b876-4d04-b122-10afd858d465.4453.14, ofs0, ns] 2024-03-04T18:02:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.861+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595088 cookie 94571568697856 notifier 5928 bl.length()=252 2024-03-04T18:02:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.873+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/797b67a4-b876-4d04-b122-10afd858d465.4453.14 : miss 2024-03-04T18:02:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.873+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.873+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.873+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/797b67a4-b876-4d04-b122-10afd858d465.4453.14 info.flags=0x0 2024-03-04T18:02:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.873+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/797b67a4-b876-4d04-b122-10afd858d465.4453.14 to cache LRU end 2024-03-04T18:02:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.877+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:797b67a4-b876-4d04-b122-10afd858d465.4453.13 : miss 2024-03-04T18:02:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.877+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.877+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=402 2024-03-04T18:02:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.877+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:797b67a4-b876-4d04-b122-10afd858d465.4453.13 info.flags=0x17 2024-03-04T18:02:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.877+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:797b67a4-b876-4d04-b122-10afd858d465.4453.13 to cache LRU end 2024-03-04T18:02:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.877+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.877+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.877+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:797b67a4-b876-4d04-b122-10afd858d465.4453.13 2024-03-04T18:02:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.881+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-03-04T18:02:41.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.881+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-03-04T18:02:41.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.885+0000 7f41a76dd980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-03-04T18:02:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.885+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a548b0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x560328c68448 s->prefetch_data=0 2024-03-04T18:02:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.889+0000 7f41a76dd980 10 manifest: total_size = 10485760 2024-03-04T18:02:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.889+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.13624126864927501956 2024-03-04T18:02:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.889+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a548b0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x560328c68448 s->prefetch_data=0 2024-03-04T18:02:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.889+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c68448 2024-03-04T18:02:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.889+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.13.3 2024-03-04T18:02:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.13624126864927501956 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.1wJF164StEx4we8RN9joPNXbJudUpyq_0 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 338 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.1wJF164StEx4we8RN9joPNXbJudUpyq_1 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 575 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.1wJF164StEx4we8RN9joPNXbJudUpyq_2 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 812 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.13624126864927501956 2024-03-04T18:02:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.893+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-03-04T18:02:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.897+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x560328c68448 s->prefetch_data=0 2024-03-04T18:02:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.897+0000 7f41a76dd980 10 manifest: total_size = 649292 2024-03-04T18:02:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.897+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.2710321245979066880 2024-03-04T18:02:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.897+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x560328c68448 s->prefetch_data=0 2024-03-04T18:02:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.897+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c68448 2024-03-04T18:02:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.897+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.13.5 2024-03-04T18:02:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.901+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2c50 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x560328c68448 s->prefetch_data=0 2024-03-04T18:02:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.901+0000 7f41a76dd980 10 manifest: total_size = 10485760 2024-03-04T18:02:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.901+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.5446563486005080308 2024-03-04T18:02:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.901+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b2c50 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x560328c68448 s->prefetch_data=0 2024-03-04T18:02:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.901+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c68448 2024-03-04T18:02:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.901+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.13.7 2024-03-04T18:02:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.905+0000 7f4077fff640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-03-04T18:02:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=4190208 stripe_ofs=4190208 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=8380416 stripe_ofs=8380416 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.5446563486005080308 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.bP1m0jH0-aUQnN_ObtJHiBgKjM4PSM3_0 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 339 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.bP1m0jH0-aUQnN_ObtJHiBgKjM4PSM3_1 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 578 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.bP1m0jH0-aUQnN_ObtJHiBgKjM4PSM3_2 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 817 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:02:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: 797b67a4-b876-4d04-b122-10afd858d465.4453.5446563486005080308 2024-03-04T18:02:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-03-04T18:02:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.909+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.913+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.913+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.913+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.913+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.913+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[797b67a4-b876-4d04-b122-10afd858d465.4453.13])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.917+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2024-03-04T18:02:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.917+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.917+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=228 2024-03-04T18:02:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.917+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x11 2024-03-04T18:02:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.917+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2024-03-04T18:02:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.921+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2024-03-04T18:02:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.921+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1, ofs0, ns] 2024-03-04T18:02:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.921+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595096 cookie 94571568746912 notifier 5928 bl.length()=184 2024-03-04T18:02:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.925+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:797b67a4-b876-4d04-b122-10afd858d465.4453.13 from cache 2024-03-04T18:02:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.925+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:797b67a4-b876-4d04-b122-10afd858d465.4453.13, ofs0, ns] 2024-03-04T18:02:41.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.925+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595101 cookie 94571568760400 notifier 5928 bl.length()=242 2024-03-04T18:02:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.933+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/797b67a4-b876-4d04-b122-10afd858d465.4453.13 : miss 2024-03-04T18:02:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.933+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.933+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.933+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/797b67a4-b876-4d04-b122-10afd858d465.4453.13 info.flags=0x0 2024-03-04T18:02:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.933+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/797b67a4-b876-4d04-b122-10afd858d465.4453.13 to cache LRU end 2024-03-04T18:02:41.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.941+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:797b67a4-b876-4d04-b122-10afd858d465.4453.6 : miss 2024-03-04T18:02:41.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.941+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.941+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=397 2024-03-04T18:02:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.941+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:797b67a4-b876-4d04-b122-10afd858d465.4453.6 info.flags=0x17 2024-03-04T18:02:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.941+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:797b67a4-b876-4d04-b122-10afd858d465.4453.6 to cache LRU end 2024-03-04T18:02:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.941+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.941+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:41.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.941+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:797b67a4-b876-4d04-b122-10afd858d465.4453.6 2024-03-04T18:02:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.941+0000 7f41a76dd980 10 cls_bucket_list_unordered: got ___ 2024-03-04T18:02:41.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.945+0000 7f41a76dd980 10 cls_bucket_list_unordered: got __ _ 2024-03-04T18:02:41.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.945+0000 7f41a76dd980 10 cls_bucket_list_unordered: got __ 2024-03-04T18:02:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.949+0000 7f41a76dd980 10 cls_bucket_list_unordered: got obj 2024-03-04T18:02:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.949+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x560328c64bf8 s->prefetch_data=0 2024-03-04T18:02:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.949+0000 7f41a76dd980 10 manifest: total_size = 0 2024-03-04T18:02:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.949+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.11818508825387073697 2024-03-04T18:02:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.949+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x560328c64bf8 s->prefetch_data=0 2024-03-04T18:02:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.949+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c64bf8 2024-03-04T18:02:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.949+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.6.2 2024-03-04T18:02:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.953+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419805be50 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x560328c64bf8 s->prefetch_data=0 2024-03-04T18:02:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.953+0000 7f41a76dd980 10 manifest: total_size = 0 2024-03-04T18:02:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.953+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.11197886365613149349 2024-03-04T18:02:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.953+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419805be50 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x560328c64bf8 s->prefetch_data=0 2024-03-04T18:02:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.953+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c64bf8 2024-03-04T18:02:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.953+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.6.3 2024-03-04T18:02:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.957+0000 7f4077fff640 20 handle_completion(): completion ok for obj=___ 2024-03-04T18:02:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.961+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x560328c64bf8 s->prefetch_data=0 2024-03-04T18:02:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.961+0000 7f41a76dd980 10 manifest: total_size = 0 2024-03-04T18:02:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.961+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.18408449177756930788 2024-03-04T18:02:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.961+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41980693b0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x560328c64bf8 s->prefetch_data=0 2024-03-04T18:02:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.961+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c64bf8 2024-03-04T18:02:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.961+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.6.6 2024-03-04T18:02:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.961+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=__ _ 2024-03-04T18:02:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.969+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c05c290 obj=test-client.0-r-test-small-obj-data.1:obj state=0x560328c64bf8 s->prefetch_data=0 2024-03-04T18:02:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.969+0000 7f41a76dd980 10 manifest: total_size = 0 2024-03-04T18:02:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.969+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.14275043276068586101 2024-03-04T18:02:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.969+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419c05c290 obj=test-client.0-r-test-small-obj-data.1:obj state=0x560328c64bf8 s->prefetch_data=0 2024-03-04T18:02:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.969+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c64bf8 2024-03-04T18:02:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.969+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.6.8 2024-03-04T18:02:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.969+0000 7f4077fff640 20 handle_completion(): completion ok for obj=__ 2024-03-04T18:02:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.973+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.973+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.977+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=obj 2024-03-04T18:02:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.977+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.977+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.977+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.977+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.977+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[797b67a4-b876-4d04-b122-10afd858d465.4453.6])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.981+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2024-03-04T18:02:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.981+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.981+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=223 2024-03-04T18:02:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.981+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x11 2024-03-04T18:02:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.981+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2024-03-04T18:02:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.985+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2024-03-04T18:02:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.985+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.1, ofs0, ns] 2024-03-04T18:02:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.985+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595189 cookie 94571568742720 notifier 5928 bl.length()=181 2024-03-04T18:02:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.989+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:797b67a4-b876-4d04-b122-10afd858d465.4453.6 from cache 2024-03-04T18:02:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.989+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:797b67a4-b876-4d04-b122-10afd858d465.4453.6, ofs0, ns] 2024-03-04T18:02:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.989+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595091 cookie 94571568700064 notifier 5928 bl.length()=238 2024-03-04T18:02:41.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.997+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/797b67a4-b876-4d04-b122-10afd858d465.4453.6 : miss 2024-03-04T18:02:41.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.997+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.997+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:41.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.997+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/797b67a4-b876-4d04-b122-10afd858d465.4453.6 info.flags=0x0 2024-03-04T18:02:41.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:41.997+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/797b67a4-b876-4d04-b122-10afd858d465.4453.6 to cache LRU end 2024-03-04T18:02:41.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.005+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:797b67a4-b876-4d04-b122-10afd858d465.4453.7 : miss 2024-03-04T18:02:41.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.005+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.005+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=397 2024-03-04T18:02:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.005+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:797b67a4-b876-4d04-b122-10afd858d465.4453.7 info.flags=0x17 2024-03-04T18:02:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.005+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:797b67a4-b876-4d04-b122-10afd858d465.4453.7 to cache LRU end 2024-03-04T18:02:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.005+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:41.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.005+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:42.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.005+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:797b67a4-b876-4d04-b122-10afd858d465.4453.7 2024-03-04T18:02:42.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.005+0000 7f41a76dd980 10 cls_bucket_list_unordered: got ___ 2024-03-04T18:02:42.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.009+0000 7f41a76dd980 10 cls_bucket_list_unordered: got __ _ 2024-03-04T18:02:42.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.009+0000 7f41a76dd980 10 cls_bucket_list_unordered: got __ 2024-03-04T18:02:42.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.013+0000 7f41a76dd980 10 cls_bucket_list_unordered: got obj 2024-03-04T18:02:42.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.013+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a54930 obj=test-client.0-r-test-small-obj-data.2:___ state=0x560328a53008 s->prefetch_data=0 2024-03-04T18:02:42.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.013+0000 7f41a76dd980 10 manifest: total_size = 524288 2024-03-04T18:02:42.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.013+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.1606604909760334704 2024-03-04T18:02:42.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.013+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a54930 obj=test-client.0-r-test-small-obj-data.2:___ state=0x560328a53008 s->prefetch_data=0 2024-03-04T18:02:42.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.013+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328a53008 2024-03-04T18:02:42.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.013+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.7.2 2024-03-04T18:02:42.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.021+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a54930 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x560328a53008 s->prefetch_data=0 2024-03-04T18:02:42.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.021+0000 7f41a76dd980 10 manifest: total_size = 24595 2024-03-04T18:02:42.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.021+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.15858508486191336661 2024-03-04T18:02:42.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.021+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a54930 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x560328a53008 s->prefetch_data=0 2024-03-04T18:02:42.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.021+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328a53008 2024-03-04T18:02:42.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.021+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.7.3 2024-03-04T18:02:42.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.021+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=___ 2024-03-04T18:02:42.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.025+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a54930 obj=test-client.0-r-test-small-obj-data.2:__ state=0x560328a53008 s->prefetch_data=0 2024-03-04T18:02:42.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.029+0000 7f41a76dd980 10 manifest: total_size = 38 2024-03-04T18:02:42.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.029+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.8620468553927254072 2024-03-04T18:02:42.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.029+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a54930 obj=test-client.0-r-test-small-obj-data.2:__ state=0x560328a53008 s->prefetch_data=0 2024-03-04T18:02:42.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.029+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328a53008 2024-03-04T18:02:42.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.029+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.7.6 2024-03-04T18:02:42.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.029+0000 7f4077fff640 20 handle_completion(): completion ok for obj=__ _ 2024-03-04T18:02:42.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.033+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a54930 obj=test-client.0-r-test-small-obj-data.2:obj state=0x560328a53008 s->prefetch_data=0 2024-03-04T18:02:42.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.033+0000 7f41a76dd980 10 manifest: total_size = 24595 2024-03-04T18:02:42.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.033+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.13224149212508047905 2024-03-04T18:02:42.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.033+0000 7f41a76dd980 20 get_obj_state: octx=0x560328a54930 obj=test-client.0-r-test-small-obj-data.2:obj state=0x560328a53008 s->prefetch_data=0 2024-03-04T18:02:42.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.033+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328a53008 2024-03-04T18:02:42.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.033+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.7.8 2024-03-04T18:02:42.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.037+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=__ 2024-03-04T18:02:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.041+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:42.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.041+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:42.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.045+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=obj 2024-03-04T18:02:42.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.045+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:42.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.045+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:42.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.045+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:42.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.045+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:42.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.045+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[797b67a4-b876-4d04-b122-10afd858d465.4453.7])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:42.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.049+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2024-03-04T18:02:42.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.049+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.049+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=223 2024-03-04T18:02:42.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.049+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x11 2024-03-04T18:02:42.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.049+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2024-03-04T18:02:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.053+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2024-03-04T18:02:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.053+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.2, ofs0, ns] 2024-03-04T18:02:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.053+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595190 cookie 94571568742720 notifier 5928 bl.length()=181 2024-03-04T18:02:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.057+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:797b67a4-b876-4d04-b122-10afd858d465.4453.7 from cache 2024-03-04T18:02:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.057+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:797b67a4-b876-4d04-b122-10afd858d465.4453.7, ofs0, ns] 2024-03-04T18:02:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.057+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595092 cookie 94571568700064 notifier 5928 bl.length()=238 2024-03-04T18:02:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.065+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/797b67a4-b876-4d04-b122-10afd858d465.4453.7 : miss 2024-03-04T18:02:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.065+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.069+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.069+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/797b67a4-b876-4d04-b122-10afd858d465.4453.7 info.flags=0x0 2024-03-04T18:02:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.069+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/797b67a4-b876-4d04-b122-10afd858d465.4453.7 to cache LRU end 2024-03-04T18:02:42.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.073+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:797b67a4-b876-4d04-b122-10afd858d465.4453.8 : miss 2024-03-04T18:02:42.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.073+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.073+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=397 2024-03-04T18:02:42.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.073+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:797b67a4-b876-4d04-b122-10afd858d465.4453.8 info.flags=0x17 2024-03-04T18:02:42.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.073+0000 7f41a76dd980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:797b67a4-b876-4d04-b122-10afd858d465.4453.8 to cache LRU end 2024-03-04T18:02:42.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.073+0000 7f41a76dd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:42.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.073+0000 7f41a76dd980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:02:42.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.073+0000 7f41a76dd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:797b67a4-b876-4d04-b122-10afd858d465.4453.8 2024-03-04T18:02:42.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.077+0000 7f41a76dd980 10 cls_bucket_list_unordered: got ___ 2024-03-04T18:02:42.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.077+0000 7f41a76dd980 10 cls_bucket_list_unordered: got __ _ 2024-03-04T18:02:42.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.077+0000 7f41a76dd980 10 cls_bucket_list_unordered: got __ 2024-03-04T18:02:42.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.081+0000 7f41a76dd980 10 cls_bucket_list_unordered: got obj 2024-03-04T18:02:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.081+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419807efc0 obj=test-client.0-r-test-small-obj-data.3:___ state=0x560328c95548 s->prefetch_data=0 2024-03-04T18:02:42.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.081+0000 7f41a76dd980 10 manifest: total_size = 49187 2024-03-04T18:02:42.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.081+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.9124774503537909260 2024-03-04T18:02:42.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.081+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419807efc0 obj=test-client.0-r-test-small-obj-data.3:___ state=0x560328c95548 s->prefetch_data=0 2024-03-04T18:02:42.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.081+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c95548 2024-03-04T18:02:42.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.081+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.8.2 2024-03-04T18:02:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.089+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419807efc0 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x560328c95548 s->prefetch_data=0 2024-03-04T18:02:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.089+0000 7f41a76dd980 10 manifest: total_size = 1048576 2024-03-04T18:02:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.089+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.18239843232747866007 2024-03-04T18:02:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.089+0000 7f41a76dd980 20 get_obj_state: octx=0x7f419807efc0 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x560328c95548 s->prefetch_data=0 2024-03-04T18:02:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.089+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c95548 2024-03-04T18:02:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.089+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.8.3 2024-03-04T18:02:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.093+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=___ 2024-03-04T18:02:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.093+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b54f0 obj=test-client.0-r-test-small-obj-data.3:__ state=0x560328c95548 s->prefetch_data=0 2024-03-04T18:02:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.097+0000 7f41a76dd980 10 manifest: total_size = 49187 2024-03-04T18:02:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.097+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.8050630702416608813 2024-03-04T18:02:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.097+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b54f0 obj=test-client.0-r-test-small-obj-data.3:__ state=0x560328c95548 s->prefetch_data=0 2024-03-04T18:02:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.097+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c95548 2024-03-04T18:02:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.097+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.8.6 2024-03-04T18:02:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.097+0000 7f4077fff640 20 handle_completion(): completion ok for obj=__ _ 2024-03-04T18:02:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.101+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b54f0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x560328c95548 s->prefetch_data=0 2024-03-04T18:02:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.101+0000 7f41a76dd980 10 manifest: total_size = 1048576 2024-03-04T18:02:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.101+0000 7f41a76dd980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.2369260008643726306 2024-03-04T18:02:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.101+0000 7f41a76dd980 20 get_obj_state: octx=0x7f41a00b54f0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x560328c95548 s->prefetch_data=0 2024-03-04T18:02:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.101+0000 7f41a76dd980 20 prepare_atomic_modification: state is not atomic. state=0x560328c95548 2024-03-04T18:02:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.101+0000 7f41a76dd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.8.8 2024-03-04T18:02:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.101+0000 7f4088ff9640 20 handle_completion(): completion ok for obj=__ 2024-03-04T18:02:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.105+0000 7f41a76dd980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:02:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.105+0000 7f41a76dd980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-04T18:02:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.109+0000 7f4077fff640 20 handle_completion(): completion ok for obj=obj 2024-03-04T18:02:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.109+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:02:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.109+0000 7f41a76dd980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:02:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.109+0000 7f41a76dd980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:02:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.109+0000 7f41a76dd980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:02:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.109+0000 7f41a76dd980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[797b67a4-b876-4d04-b122-10afd858d465.4453.8])"; uploads.size()=0; is_truncated=0 2024-03-04T18:02:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.113+0000 7f41a76dd980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2024-03-04T18:02:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.113+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.113+0000 7f41a76dd980 20 rados_obj.operate() r=0 bl.length=223 2024-03-04T18:02:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.113+0000 7f41a76dd980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x11 2024-03-04T18:02:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.113+0000 7f41a76dd980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2024-03-04T18:02:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.117+0000 7f41a76dd980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2024-03-04T18:02:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.117+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.3, ofs0, ns] 2024-03-04T18:02:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.117+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595191 cookie 94571568742720 notifier 5928 bl.length()=181 2024-03-04T18:02:42.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.121+0000 7f41a76dd980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:797b67a4-b876-4d04-b122-10afd858d465.4453.8 from cache 2024-03-04T18:02:42.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.121+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:797b67a4-b876-4d04-b122-10afd858d465.4453.8, ofs0, ns] 2024-03-04T18:02:42.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.121+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595093 cookie 94571568700064 notifier 5928 bl.length()=238 2024-03-04T18:02:42.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.129+0000 7f41a76dd980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/797b67a4-b876-4d04-b122-10afd858d465.4453.8 : miss 2024-03-04T18:02:42.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.129+0000 7f41a76dd980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.133+0000 7f41a76dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.133+0000 7f41a76dd980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/797b67a4-b876-4d04-b122-10afd858d465.4453.8 info.flags=0x0 2024-03-04T18:02:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.133+0000 7f41a76dd980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/797b67a4-b876-4d04-b122-10afd858d465.4453.8 to cache LRU end 2024-03-04T18:02:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.137+0000 7f41a76dd980 10 removing key index: RRNEBMPDKTWKEMLCMFII 2024-03-04T18:02:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.137+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.keys:RRNEBMPDKTWKEMLCMFII, ofs0, ns] 2024-03-04T18:02:42.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.137+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595102 cookie 94571568757760 notifier 5928 bl.length()=170 2024-03-04T18:02:42.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.141+0000 7f41a76dd980 10 removing email index: ragweed.client.0+test@test.test 2024-03-04T18:02:42.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.145+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-03-04T18:02:42.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.145+0000 7f4077fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595097 cookie 94571568746912 notifier 5928 bl.length()=182 2024-03-04T18:02:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.145+0000 7f41a76dd980 10 removing user buckets index 2024-03-04T18:02:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.149+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0.buckets, ofs0, ns] 2024-03-04T18:02:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.149+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595103 cookie 94571568754160 notifier 5928 bl.length()=173 2024-03-04T18:02:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.153+0000 7f41a76dd980 10 removing user index: ragweed.client.0 2024-03-04T18:02:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.153+0000 7f41a76dd980 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2024-03-04T18:02:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.153+0000 7f41a76dd980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-03-04T18:02:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.153+0000 7f4088ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595192 cookie 94571568749856 notifier 5928 bl.length()=165 2024-03-04T18:02:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.161+0000 7f405bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:02:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.165+0000 7f41a76dd980 20 remove_watcher() i=0 2024-03-04T18:02:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.165+0000 7f41a76dd980 2 removed watcher, disabling cache 2024-03-04T18:02:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.169+0000 7f41a76dd980 20 remove_watcher() i=1 2024-03-04T18:02:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.169+0000 7f41a76dd980 20 remove_watcher() i=2 2024-03-04T18:02:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.173+0000 7f41a76dd980 20 remove_watcher() i=3 2024-03-04T18:02:42.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.177+0000 7f41a76dd980 20 remove_watcher() i=4 2024-03-04T18:02:42.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.177+0000 7f41a76dd980 20 remove_watcher() i=5 2024-03-04T18:02:42.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.181+0000 7f41a76dd980 20 remove_watcher() i=6 2024-03-04T18:02:42.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.185+0000 7f41a76dd980 20 remove_watcher() i=7 2024-03-04T18:02:42.197 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2024-03-04T18:02:42.279 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:02:42.279 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:02:42.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 realm 2024-03-04T18:02:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:02:42.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:42.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:02:42.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.329+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.333+0000 7f7e661bc980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:02:42.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.333+0000 7f7e661bc980 20 searching for the correct realm 2024-03-04T18:02:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.341+0000 7f7e644d5640 0 -- 172.21.15.47:0/3846319578 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55b81f117400 msgr2=0x55b81f1197f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:02:42.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.345+0000 7f7e634d3640 0 -- 172.21.15.47:0/3846319578 >> [v2:172.21.15.152:6808/929675582,v1:172.21.15.152:6809/929675582] conn(0x55b81f120910 msgr2=0x55b81f122dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:02:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:02:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:02:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:02:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:02:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:02:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:02:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:02:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:02:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 zone default found 2024-03-04T18:02:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 4 Realm: () 2024-03-04T18:02:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:02:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:02:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:02:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 zonegroup default 2024-03-04T18:02:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.353+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.357+0000 7f7e661bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.357+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.357+0000 7f7e661bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:02:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.357+0000 7f7e661bc980 20 started sync module instance, tier type = 2024-03-04T18:02:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.357+0000 7f7e661bc980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:02:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.373+0000 7f7e661bc980 20 add_watcher() i=0 2024-03-04T18:02:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.373+0000 7f7e661bc980 20 add_watcher() i=1 2024-03-04T18:02:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.373+0000 7f7e661bc980 20 add_watcher() i=2 2024-03-04T18:02:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.373+0000 7f7e661bc980 20 add_watcher() i=3 2024-03-04T18:02:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.373+0000 7f7e661bc980 20 add_watcher() i=4 2024-03-04T18:02:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.373+0000 7f7e661bc980 20 add_watcher() i=5 2024-03-04T18:02:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.373+0000 7f7e661bc980 20 add_watcher() i=6 2024-03-04T18:02:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.377+0000 7f7e661bc980 20 add_watcher() i=7 2024-03-04T18:02:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.377+0000 7f7e661bc980 2 all 8 watchers are set, enabling cache 2024-03-04T18:02:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.381+0000 7f7d19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:02:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.381+0000 7f7e661bc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:02:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.381+0000 7f7e661bc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.457+0000 7f7d18fe1640 20 reqs_thread_entry: start 2024-03-04T18:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.457+0000 7f7e661bc980 20 init_complete bucket index max shards: 11 2024-03-04T18:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.457+0000 7f7e661bc980 20 Filter name: none 2024-03-04T18:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.461+0000 7f7c927fc640 20 reqs_thread_entry: start 2024-03-04T18:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.461+0000 7f7e661bc980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-03-04T18:02:42.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.461+0000 7f7e661bc980 20 rados->read ofs=0 len=0 2024-03-04T18:02:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.461+0000 7f7e661bc980 20 rados_obj.operate() r=0 bl.length=453 2024-03-04T18:02:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.461+0000 7f7e661bc980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x13 2024-03-04T18:02:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.461+0000 7f7e661bc980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-03-04T18:02:42.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.461+0000 7f7e661bc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:02:42.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.461+0000 7f7e661bc980 10 removing key index: WIJRHVIFQUKMQINMYKYW 2024-03-04T18:02:42.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.465+0000 7f7e661bc980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.keys:WIJRHVIFQUKMQINMYKYW, ofs0, ns] 2024-03-04T18:02:42.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.465+0000 7f7d42ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595104 cookie 94249283587312 notifier 5612 bl.length()=170 2024-03-04T18:02:42.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.465+0000 7f7e661bc980 10 removing email index: sysuser.client.0+test@test.test 2024-03-04T18:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.469+0000 7f7e661bc980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-03-04T18:02:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.469+0000 7f7d42ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595193 cookie 94249283625056 notifier 5612 bl.length()=182 2024-03-04T18:02:42.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.473+0000 7f7e661bc980 10 removing user buckets index 2024-03-04T18:02:42.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.473+0000 7f7e661bc980 10 removing user index: sysuser.client.0 2024-03-04T18:02:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.477+0000 7f7e661bc980 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2024-03-04T18:02:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.477+0000 7f7e661bc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-03-04T18:02:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.477+0000 7f7d42ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595098 cookie 94249283585184 notifier 5612 bl.length()=165 2024-03-04T18:02:42.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.481+0000 7f7d19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:02:42.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.485+0000 7f7e661bc980 20 remove_watcher() i=0 2024-03-04T18:02:42.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.485+0000 7f7e661bc980 2 removed watcher, disabling cache 2024-03-04T18:02:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.489+0000 7f7e661bc980 20 remove_watcher() i=1 2024-03-04T18:02:42.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.489+0000 7f7e661bc980 20 remove_watcher() i=2 2024-03-04T18:02:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.493+0000 7f7e661bc980 20 remove_watcher() i=3 2024-03-04T18:02:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.493+0000 7f7e661bc980 20 remove_watcher() i=4 2024-03-04T18:02:42.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.497+0000 7f7e661bc980 20 remove_watcher() i=5 2024-03-04T18:02:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.501+0000 7f7e661bc980 20 remove_watcher() i=6 2024-03-04T18:02:42.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:02:42.501+0000 7f7e661bc980 20 remove_watcher() i=7 2024-03-04T18:02:42.516 INFO:tasks.ragweed:Removing ragweed... 2024-03-04T18:02:42.516 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/ragweed.client.0 2024-03-04T18:02:42.611 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T18:02:42.622 INFO:tasks.workunit:Pulling workunits from ref 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T18:02:42.622 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T18:02:42.623 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T18:02:42.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T18:02:42.630 INFO:teuthology.orchestra.run.smithi047.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T18:02:42.630 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T18:02:42.678 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T18:02:42.678 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T18:02:42.727 INFO:tasks.workunit:timeout=3h 2024-03-04T18:02:42.727 INFO:tasks.workunit:cleanup=True 2024-03-04T18:02:42.727 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/pritha-srivastava/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T18:02:42.776 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T18:03:21.254 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to '3aec390336fdfdc8f927818b718cfb0361bd47c8'. 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-03-04T18:03:21.255 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:21.256 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-03-04T18:03:21.256 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:21.256 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T18:03:21.256 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:21.256 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at 3aec390336f qa/d4n: Start redis up before rgw 2024-03-04T18:03:21.261 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T18:03:21.309 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T18:03:21.311 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T18:03:21.311 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T18:03:21.366 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T18:03:21.423 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T18:03:21.463 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T18:03:21.465 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T18:03:21.465 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T18:03:21.510 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T18:03:21.514 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T18:03:21.514 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T18:03:21.559 INFO:tasks.workunit:Running workunits matching rgw/run-reshard.sh on client.0... 2024-03-04T18:03:21.560 INFO:tasks.workunit:Running workunit rgw/run-reshard.sh... 2024-03-04T18:03:21.560 DEBUG:teuthology.orchestra.run.smithi047:workunit test rgw/run-reshard.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-03-04T18:03:21.615 INFO:tasks.workunit.client.0.smithi047.stderr:+ ulimit -c 0 2024-03-04T18:03:21.615 INFO:tasks.workunit.client.0.smithi047.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-03-04T18:03:21.616 INFO:tasks.workunit.client.0.smithi047.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T18:03:21.616 INFO:tasks.workunit.client.0.smithi047.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T18:03:24.594 INFO:tasks.workunit.client.0.smithi047.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-03-04T18:03:24.594 INFO:tasks.workunit.client.0.smithi047.stderr:++ deactivate nondestructive 2024-03-04T18:03:24.594 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n '' ']' 2024-03-04T18:03:24.594 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n '' ']' 2024-03-04T18:03:24.594 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T18:03:24.594 INFO:tasks.workunit.client.0.smithi047.stderr:++ hash -r 2024-03-04T18:03:24.594 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n '' ']' 2024-03-04T18:03:24.594 INFO:tasks.workunit.client.0.smithi047.stderr:++ unset VIRTUAL_ENV 2024-03-04T18:03:24.594 INFO:tasks.workunit.client.0.smithi047.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ export VIRTUAL_ENV 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ export PATH 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n '' ']' 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -z '' ']' 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ _OLD_VIRTUAL_PS1= 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ PS1='(rgw) ' 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ export PS1 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ export VIRTUAL_ENV_PROMPT 2024-03-04T18:03:24.595 INFO:tasks.workunit.client.0.smithi047.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T18:03:24.596 INFO:tasks.workunit.client.0.smithi047.stderr:++ hash -r 2024-03-04T18:03:24.596 INFO:tasks.workunit.client.0.smithi047.stderr:+ pip install pip --upgrade 2024-03-04T18:03:24.963 INFO:tasks.workunit.client.0.smithi047.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-03-04T18:03:25.177 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting pip 2024-03-04T18:03:25.184 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-03-04T18:03:25.222 INFO:tasks.workunit.client.0.smithi047.stdout:Installing collected packages: pip 2024-03-04T18:03:25.222 INFO:tasks.workunit.client.0.smithi047.stdout: Attempting uninstall: pip 2024-03-04T18:03:25.223 INFO:tasks.workunit.client.0.smithi047.stdout: Found existing installation: pip 22.0.2 2024-03-04T18:03:25.493 INFO:tasks.workunit.client.0.smithi047.stdout: Uninstalling pip-22.0.2: 2024-03-04T18:03:25.499 INFO:tasks.workunit.client.0.smithi047.stdout: Successfully uninstalled pip-22.0.2 2024-03-04T18:03:26.467 INFO:tasks.workunit.client.0.smithi047.stdout:Successfully installed pip-24.0 2024-03-04T18:03:26.526 INFO:tasks.workunit.client.0.smithi047.stderr:+ pip install boto3 2024-03-04T18:03:27.518 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting boto3 2024-03-04T18:03:27.519 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached boto3-1.34.54-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T18:03:28.067 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting botocore<1.35.0,>=1.34.54 (from boto3) 2024-03-04T18:03:28.068 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached botocore-1.34.54-py3-none-any.whl.metadata (5.7 kB) 2024-03-04T18:03:28.137 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-03-04T18:03:28.138 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-03-04T18:03:28.206 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-03-04T18:03:28.207 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached s3transfer-0.10.0-py3-none-any.whl.metadata (1.7 kB) 2024-03-04T18:03:28.291 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T18:03:28.293 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-03-04T18:03:28.421 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting urllib3<2.1,>=1.25.4 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T18:03:28.422 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T18:03:28.584 INFO:tasks.workunit.client.0.smithi047.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T18:03:28.585 INFO:tasks.workunit.client.0.smithi047.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-03-04T18:03:28.601 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached boto3-1.34.54-py3-none-any.whl (139 kB) 2024-03-04T18:03:28.603 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached botocore-1.34.54-py3-none-any.whl (12.0 MB) 2024-03-04T18:03:28.629 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-03-04T18:03:28.631 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached s3transfer-0.10.0-py3-none-any.whl (82 kB) 2024-03-04T18:03:28.632 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-03-04T18:03:28.634 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached urllib3-2.0.7-py3-none-any.whl (124 kB) 2024-03-04T18:03:28.636 INFO:tasks.workunit.client.0.smithi047.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-03-04T18:03:28.738 INFO:tasks.workunit.client.0.smithi047.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-03-04T18:03:29.899 INFO:tasks.workunit.client.0.smithi047.stdout:Successfully installed boto3-1.34.54 botocore-1.34.54 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.0 six-1.16.0 urllib3-2.0.7 2024-03-04T18:03:30.062 INFO:tasks.workunit.client.0.smithi047.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_reshard.py 2024-03-04T18:03:30.203 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin user create --uid tester --display-name "Testing" --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn 2024-03-04T18:03:30.242 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:30.242 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:30.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.291+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 realm 2024-03-04T18:03:30.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:30.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:30.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:30.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.295+0000 7f1863d61980 20 searching for the correct realm 2024-03-04T18:03:30.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.303+0000 7f186207a640 0 -- 172.21.15.47:0/2280091069 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f1858021470 msgr2=0x7f1858021890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:30.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.307+0000 7f1861879640 0 -- 172.21.15.47:0/2280091069 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f185c0f9bc0 msgr2=0x7f185c109120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:30.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.315+0000 7f1863d61980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:30.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.315+0000 7f1863d61980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:30.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.315+0000 7f1863d61980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:30.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.315+0000 7f1863d61980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:30.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.315+0000 7f1863d61980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:30.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.315+0000 7f1863d61980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:30.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.315+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.315 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:30.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:30.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 zone default found 2024-03-04T18:03:30.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 4 Realm: () 2024-03-04T18:03:30.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:30.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:30.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:30.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 zonegroup default 2024-03-04T18:03:30.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 started sync module instance, tier type = 2024-03-04T18:03:30.317 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.319+0000 7f1863d61980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:30.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.339+0000 7f1863d61980 20 add_watcher() i=0 2024-03-04T18:03:30.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.339+0000 7f1863d61980 20 add_watcher() i=1 2024-03-04T18:03:30.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.339+0000 7f1863d61980 20 add_watcher() i=2 2024-03-04T18:03:30.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.339+0000 7f1863d61980 20 add_watcher() i=3 2024-03-04T18:03:30.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.339+0000 7f1863d61980 20 add_watcher() i=4 2024-03-04T18:03:30.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.339+0000 7f1863d61980 20 add_watcher() i=5 2024-03-04T18:03:30.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.339+0000 7f1863d61980 20 add_watcher() i=6 2024-03-04T18:03:30.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.343+0000 7f1863d61980 20 add_watcher() i=7 2024-03-04T18:03:30.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.343+0000 7f1863d61980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:30.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.343+0000 7f17137fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:30.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:30.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:30.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 5 note: GC not initialized 2024-03-04T18:03:30.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f17127fc640 20 reqs_thread_entry: start 2024-03-04T18:03:30.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:30.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 20 Filter name: none 2024-03-04T18:03:30.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1683fff640 20 reqs_thread_entry: start 2024-03-04T18:03:30.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 10 cache get: name=default.rgw.meta+users.uid+tester : miss 2024-03-04T18:03:30.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x0 2024-03-04T18:03:30.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 10 adding default.rgw.meta+users.uid+tester to cache LRU end 2024-03-04T18:03:30.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-03-04T18:03:30.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-03-04T18:03:30.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-04T18:03:30.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-03-04T18:03:30.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.347+0000 7f1863d61980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-03-04T18:03:30.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.351+0000 7f1863d61980 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x17 2024-03-04T18:03:30.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.351+0000 7f1863d61980 10 moving default.rgw.meta+users.uid+tester to cache LRU end 2024-03-04T18:03:30.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.351+0000 7f1863d61980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester, ofs0, ns] 2024-03-04T18:03:30.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.351+0000 7f17457fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595105 cookie 94904978435136 notifier 5964 bl.length()=536 2024-03-04T18:03:30.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.351+0000 7f17457fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester info.flags=0x17 2024-03-04T18:03:30.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.351+0000 7f17457fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester to cache LRU end 2024-03-04T18:03:30.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.359+0000 7f1863d61980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-03-04T18:03:30.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.359+0000 7f1863d61980 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-04T18:03:30.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.359+0000 7f1863d61980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-03-04T18:03:30.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.359+0000 7f1744ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595106 cookie 94904978434016 notifier 5964 bl.length()=180 2024-03-04T18:03:30.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.359+0000 7f1744ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-03-04T18:03:30.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.359+0000 7f1744ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-04T18:03:30.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.363+0000 7f17137fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:30.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.367+0000 7f1863d61980 20 remove_watcher() i=0 2024-03-04T18:03:30.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.367+0000 7f1863d61980 2 removed watcher, disabling cache 2024-03-04T18:03:30.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.367+0000 7f1863d61980 20 remove_watcher() i=1 2024-03-04T18:03:30.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.371+0000 7f1863d61980 20 remove_watcher() i=2 2024-03-04T18:03:30.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.371+0000 7f1863d61980 20 remove_watcher() i=3 2024-03-04T18:03:30.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.375+0000 7f1863d61980 20 remove_watcher() i=4 2024-03-04T18:03:30.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.379+0000 7f1863d61980 20 remove_watcher() i=5 2024-03-04T18:03:30.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.379+0000 7f1863d61980 20 remove_watcher() i=6 2024-03-04T18:03:30.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:30.383+0000 7f1863d61980 20 remove_watcher() i=7 2024-03-04T18:03:46.812 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:46.813 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:46.889 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:46.890 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:47.028 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:47.028 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:47.169 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:47.169 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:47.241 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:47.335 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:47.412 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard add 2024-03-04T18:03:47.412 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:47.412 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:03:47.446 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:47.446 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:47.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.495+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 20 realm 2024-03-04T18:03:47.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:47.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:47.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.499+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.503+0000 7f5b5a5e2980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:47.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.503+0000 7f5b5a5e2980 20 searching for the correct realm 2024-03-04T18:03:47.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.507+0000 7f5b580fa640 0 -- 172.21.15.47:0/3775661272 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f5b48002730 msgr2=0x7f5b48100cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:47.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.515+0000 7f5b580fa640 0 -- 172.21.15.47:0/3775661272 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x556e47a6ba50 msgr2=0x556e47a6df10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:47.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:47.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:47.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:47.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:47.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:47.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:47.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:47.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 zone default found 2024-03-04T18:03:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 4 Realm: () 2024-03-04T18:03:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 zonegroup default 2024-03-04T18:03:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 started sync module instance, tier type = 2024-03-04T18:03:47.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.523+0000 7f5b5a5e2980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:47.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.543+0000 7f5b5a5e2980 20 add_watcher() i=0 2024-03-04T18:03:47.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.543+0000 7f5b5a5e2980 20 add_watcher() i=1 2024-03-04T18:03:47.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.543+0000 7f5b5a5e2980 20 add_watcher() i=2 2024-03-04T18:03:47.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.543+0000 7f5b5a5e2980 20 add_watcher() i=3 2024-03-04T18:03:47.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.543+0000 7f5b5a5e2980 20 add_watcher() i=4 2024-03-04T18:03:47.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.543+0000 7f5b5a5e2980 20 add_watcher() i=5 2024-03-04T18:03:47.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.543+0000 7f5b5a5e2980 20 add_watcher() i=6 2024-03-04T18:03:47.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.547+0000 7f5b5a5e2980 20 add_watcher() i=7 2024-03-04T18:03:47.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.547+0000 7f5b5a5e2980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:47.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5a0efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:47.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:47.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:47.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 5 note: GC not initialized 2024-03-04T18:03:47.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5a0dfdb640 20 reqs_thread_entry: start 2024-03-04T18:03:47.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:47.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 Filter name: none 2024-03-04T18:03:47.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f597f7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:47.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:47.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:47.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:47.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:47.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:47.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.551+0000 7f5b5a5e2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:47.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.555+0000 7f5b5a5e2980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.555+0000 7f5b5a5e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.555+0000 7f5a0efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:47.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.559+0000 7f5b5a5e2980 20 remove_watcher() i=0 2024-03-04T18:03:47.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.559+0000 7f5b5a5e2980 2 removed watcher, disabling cache 2024-03-04T18:03:47.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.563+0000 7f5b5a5e2980 20 remove_watcher() i=1 2024-03-04T18:03:47.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.563+0000 7f5b5a5e2980 20 remove_watcher() i=2 2024-03-04T18:03:47.559 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.567+0000 7f5b5a5e2980 20 remove_watcher() i=3 2024-03-04T18:03:47.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.567+0000 7f5b5a5e2980 20 remove_watcher() i=4 2024-03-04T18:03:47.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.571+0000 7f5b5a5e2980 20 remove_watcher() i=5 2024-03-04T18:03:47.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.571+0000 7f5b5a5e2980 20 remove_watcher() i=6 2024-03-04T18:03:47.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.575+0000 7f5b5a5e2980 20 remove_watcher() i=7 2024-03-04T18:03:47.584 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:47.584 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:03:47.584 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: a-bucket id: 797b67a4-b876-4d04-b122-10afd858d465.4453.17 num_objects: 0 size_kb: 0 num_shards: 11 2024-03-04T18:03:47.584 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:47.584 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 12 2024-03-04T18:03:47.613 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:47.613 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:47.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.663+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.663+0000 7fee8cabb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.663+0000 7fee8cabb980 20 realm 2024-03-04T18:03:47.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.663+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.663+0000 7fee8cabb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.663+0000 7fee8cabb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:47.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.663+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.667+0000 7fee8cabb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.667+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.667+0000 7fee8cabb980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:47.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.667+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.667+0000 7fee8cabb980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:47.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.667+0000 7fee8cabb980 20 searching for the correct realm 2024-03-04T18:03:47.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.675+0000 7fee8a5d3640 0 -- 172.21.15.47:0/1739312006 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fee7c0d7030 msgr2=0x7fee7c001990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:47.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.679+0000 7fee8a5d3640 0 -- 172.21.15.47:0/1739312006 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fee84052e30 msgr2=0x7fee840811b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:47.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:47.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:47.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:47.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:47.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:47.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:47.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:47.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.687+0000 7fee8cabb980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:47.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 zone default found 2024-03-04T18:03:47.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 4 Realm: () 2024-03-04T18:03:47.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:47.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:47.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:47.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 zonegroup default 2024-03-04T18:03:47.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 started sync module instance, tier type = 2024-03-04T18:03:47.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.691+0000 7fee8cabb980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:47.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.707+0000 7fee8cabb980 20 add_watcher() i=0 2024-03-04T18:03:47.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.707+0000 7fee8cabb980 20 add_watcher() i=1 2024-03-04T18:03:47.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.707+0000 7fee8cabb980 20 add_watcher() i=2 2024-03-04T18:03:47.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.707+0000 7fee8cabb980 20 add_watcher() i=3 2024-03-04T18:03:47.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.707+0000 7fee8cabb980 20 add_watcher() i=4 2024-03-04T18:03:47.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.707+0000 7fee8cabb980 20 add_watcher() i=5 2024-03-04T18:03:47.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.707+0000 7fee8cabb980 20 add_watcher() i=6 2024-03-04T18:03:47.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.711+0000 7fee8cabb980 20 add_watcher() i=7 2024-03-04T18:03:47.705 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.711+0000 7fee8cabb980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:47.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fed40fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 5 note: GC not initialized 2024-03-04T18:03:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fecbb7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 20 Filter name: none 2024-03-04T18:03:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fecb97fa640 20 reqs_thread_entry: start 2024-03-04T18:03:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:03:47.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:47.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-03-04T18:03:47.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:03:47.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 : miss 2024-03-04T18:03:47.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:47.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:47.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:47.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:47.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:47.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.715+0000 7fee8cabb980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 2024-03-04T18:03:47.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.719+0000 7fed40fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:47.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.723+0000 7fee8cabb980 20 remove_watcher() i=0 2024-03-04T18:03:47.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.723+0000 7fee8cabb980 2 removed watcher, disabling cache 2024-03-04T18:03:47.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.723+0000 7fee8cabb980 20 remove_watcher() i=1 2024-03-04T18:03:47.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.727+0000 7fee8cabb980 20 remove_watcher() i=2 2024-03-04T18:03:47.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.727+0000 7fee8cabb980 20 remove_watcher() i=3 2024-03-04T18:03:47.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.731+0000 7fee8cabb980 20 remove_watcher() i=4 2024-03-04T18:03:47.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.735+0000 7fee8cabb980 20 remove_watcher() i=5 2024-03-04T18:03:47.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.735+0000 7fee8cabb980 20 remove_watcher() i=6 2024-03-04T18:03:47.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.739+0000 7fee8cabb980 20 remove_watcher() i=7 2024-03-04T18:03:47.749 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin reshard list 2024-03-04T18:03:47.773 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:47.773 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:47.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.823+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.819 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 realm 2024-03-04T18:03:47.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:47.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:47.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:47.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.827+0000 7f23b6906980 20 searching for the correct realm 2024-03-04T18:03:47.829 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.835+0000 7f23b3c1d640 0 -- 172.21.15.47:0/3699907296 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f23a8025b80 msgr2=0x7f23a801a050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:47.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.843+0000 7f23b4c1f640 0 -- 172.21.15.47:0/3699907296 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55c62e512550 msgr2=0x55c62e514960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:47.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.847+0000 7f23b6906980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:47.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.847+0000 7f23b6906980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:47.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.847+0000 7f23b6906980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:47.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.847+0000 7f23b6906980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:47.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.847+0000 7f23b6906980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:47.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.847+0000 7f23b6906980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:47.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.847+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.844 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 zone default found 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 4 Realm: () 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 zonegroup default 2024-03-04T18:03:47.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.846 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 started sync module instance, tier type = 2024-03-04T18:03:47.847 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.851+0000 7f23b6906980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:47.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.871+0000 7f23b6906980 20 add_watcher() i=0 2024-03-04T18:03:47.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.871+0000 7f23b6906980 20 add_watcher() i=1 2024-03-04T18:03:47.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.871+0000 7f23b6906980 20 add_watcher() i=2 2024-03-04T18:03:47.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.871+0000 7f23b6906980 20 add_watcher() i=3 2024-03-04T18:03:47.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.871+0000 7f23b6906980 20 add_watcher() i=4 2024-03-04T18:03:47.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.871+0000 7f23b6906980 20 add_watcher() i=5 2024-03-04T18:03:47.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.871+0000 7f23b6906980 20 add_watcher() i=6 2024-03-04T18:03:47.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.871+0000 7f23b6906980 20 add_watcher() i=7 2024-03-04T18:03:47.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.871+0000 7f23b6906980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:47.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.875+0000 7f226a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:47.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.875+0000 7f23b6906980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:47.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.875+0000 7f23b6906980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:47.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.875+0000 7f23b6906980 5 note: GC not initialized 2024-03-04T18:03:47.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.875+0000 7f22697e2640 20 reqs_thread_entry: start 2024-03-04T18:03:47.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.879+0000 7f23b6906980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:47.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.879+0000 7f23b6906980 20 Filter name: none 2024-03-04T18:03:47.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.879+0000 7f21daffd640 20 reqs_thread_entry: start 2024-03-04T18:03:47.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.891+0000 7f226a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:47.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.895+0000 7f23b6906980 20 remove_watcher() i=0 2024-03-04T18:03:47.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.895+0000 7f23b6906980 2 removed watcher, disabling cache 2024-03-04T18:03:47.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.895+0000 7f23b6906980 20 remove_watcher() i=1 2024-03-04T18:03:47.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.899+0000 7f23b6906980 20 remove_watcher() i=2 2024-03-04T18:03:47.894 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.899+0000 7f23b6906980 20 remove_watcher() i=3 2024-03-04T18:03:47.896 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.903+0000 7f23b6906980 20 remove_watcher() i=4 2024-03-04T18:03:47.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.903+0000 7f23b6906980 20 remove_watcher() i=5 2024-03-04T18:03:47.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.907+0000 7f23b6906980 20 remove_watcher() i=6 2024-03-04T18:03:47.904 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.911+0000 7f23b6906980 20 remove_watcher() i=7 2024-03-04T18:03:47.919 INFO:tasks.workunit.client.0.smithi047.stderr:bucket name a-bucket 2024-03-04T18:03:47.920 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard process 2024-03-04T18:03:47.920 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:47.920 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin reshard process 2024-03-04T18:03:47.944 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:47.944 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:47.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.995+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.995+0000 7fa00cfe9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.995+0000 7fa00cfe9980 20 realm 2024-03-04T18:03:47.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.995+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.995+0000 7fa00cfe9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.995+0000 7fa00cfe9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:47.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.995+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.999+0000 7fa00cfe9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:47.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.999+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.999+0000 7fa00cfe9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:47.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.999+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:47.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.999+0000 7fa00cfe9980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:47.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:47.999+0000 7fa00cfe9980 20 searching for the correct realm 2024-03-04T18:03:48.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.007+0000 7fa00b302640 0 -- 172.21.15.47:0/4247706890 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7fa000062380 msgr2=0x7fa00005b210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:48.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.011+0000 7fa00ab01640 0 -- 172.21.15.47:0/4247706890 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fa0040c6bf0 msgr2=0x7fa0040c5410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:48.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:48.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:48.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:48.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:48.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:48.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:48.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:48.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:48.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:48.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:48.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:48.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:48.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 zone default found 2024-03-04T18:03:48.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 4 Realm: () 2024-03-04T18:03:48.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:48.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:48.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:48.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 zonegroup default 2024-03-04T18:03:48.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.019+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:48.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.023+0000 7fa00cfe9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:48.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.023+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:48.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.023+0000 7fa00cfe9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:48.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.023+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:48.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.023+0000 7fa00cfe9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:48.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.023+0000 7fa00cfe9980 20 started sync module instance, tier type = 2024-03-04T18:03:48.016 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.023+0000 7fa00cfe9980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:48.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.039+0000 7fa00cfe9980 20 add_watcher() i=0 2024-03-04T18:03:48.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.039+0000 7fa00cfe9980 20 add_watcher() i=1 2024-03-04T18:03:48.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.039+0000 7fa00cfe9980 20 add_watcher() i=2 2024-03-04T18:03:48.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.039+0000 7fa00cfe9980 20 add_watcher() i=3 2024-03-04T18:03:48.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.039+0000 7fa00cfe9980 20 add_watcher() i=4 2024-03-04T18:03:48.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.039+0000 7fa00cfe9980 20 add_watcher() i=5 2024-03-04T18:03:48.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.039+0000 7fa00cfe9980 20 add_watcher() i=6 2024-03-04T18:03:48.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.043+0000 7fa00cfe9980 20 add_watcher() i=7 2024-03-04T18:03:48.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.043+0000 7fa00cfe9980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:48.039 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.047+0000 7f9ec17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:48.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.047+0000 7fa00cfe9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:48.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.047+0000 7fa00cfe9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:48.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.047+0000 7fa00cfe9980 5 note: GC not initialized 2024-03-04T18:03:48.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.047+0000 7f9e3bfff640 20 reqs_thread_entry: start 2024-03-04T18:03:48.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.047+0000 7fa00cfe9980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:48.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.047+0000 7fa00cfe9980 20 Filter name: none 2024-03-04T18:03:48.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.047+0000 7f9e39ffb640 20 reqs_thread_entry: start 2024-03-04T18:03:48.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.047+0000 7fa00cfe9980 20 processing logshard = reshard.0000000000 2024-03-04T18:03:48.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.051+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-03-04T18:03:48.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.051+0000 7fa00cfe9980 20 processing logshard = reshard.0000000001 2024-03-04T18:03:48.050 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.055+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-03-04T18:03:48.050 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.055+0000 7fa00cfe9980 20 processing logshard = reshard.0000000002 2024-03-04T18:03:48.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.059+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-03-04T18:03:48.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.059+0000 7fa00cfe9980 20 processing logshard = reshard.0000000003 2024-03-04T18:03:48.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.067+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-03-04T18:03:48.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.067+0000 7fa00cfe9980 20 processing logshard = reshard.0000000004 2024-03-04T18:03:48.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.071+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-03-04T18:03:48.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.071+0000 7fa00cfe9980 20 processing logshard = reshard.0000000005 2024-03-04T18:03:48.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.075+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-03-04T18:03:48.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.075+0000 7fa00cfe9980 20 processing logshard = reshard.0000000006 2024-03-04T18:03:48.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 20 process_entry resharding a-bucket 2024-03-04T18:03:48.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:03:48.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:48.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:48.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-03-04T18:03:48.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:03:48.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 : miss 2024-03-04T18:03:48.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:48.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:48.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:48.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:48.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:48.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:48.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.079+0000 7fa00cfe9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 2024-03-04T18:03:48.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.095+0000 7fa00cfe9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:48.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.095+0000 7fa00cfe9980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:48.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.095+0000 7fa00cfe9980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:48.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.095+0000 7fa00cfe9980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17, ofs0, ns] 2024-03-04T18:03:48.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.095+0000 7f9eedffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595100 cookie 94095557991728 notifier 6039 bl.length()=789 2024-03-04T18:03:48.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.095+0000 7f9eedffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:48.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.095+0000 7f9eedffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:48.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.095+0000 7f9eedffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:48.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.099+0000 7fa00cfe9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_sources= new_sources= 2024-03-04T18:03:48.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.099+0000 7fa00cfe9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential sources added=[] removed=[] 2024-03-04T18:03:48.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.099+0000 7fa00cfe9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_dests= new_dests= 2024-03-04T18:03:48.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.099+0000 7fa00cfe9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential dests added=[] removed=[] 2024-03-04T18:03:48.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.0 2024-03-04T18:03:48.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.0 2024-03-04T18:03:48.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.1 2024-03-04T18:03:48.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.1 2024-03-04T18:03:48.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.2 2024-03-04T18:03:48.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.2 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.3 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.3 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.4 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.4 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.5 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.5 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.6 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.6 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.7 2024-03-04T18:03:48.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.7 2024-03-04T18:03:48.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.8 2024-03-04T18:03:48.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.8 2024-03-04T18:03:48.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.9 2024-03-04T18:03:48.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.9 2024-03-04T18:03:48.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.10 2024-03-04T18:03:48.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.10 2024-03-04T18:03:48.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.11 2024-03-04T18:03:48.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.11 2024-03-04T18:03:48.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.0 2024-03-04T18:03:48.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.103+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1 2024-03-04T18:03:48.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.107+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2 2024-03-04T18:03:48.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.107+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.3 2024-03-04T18:03:48.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.107+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.4 2024-03-04T18:03:48.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.107+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.5 2024-03-04T18:03:48.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.107+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.6 2024-03-04T18:03:48.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.111+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.7 2024-03-04T18:03:48.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.111+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.8 2024-03-04T18:03:48.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.111+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.9 2024-03-04T18:03:48.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.111+0000 7fa00cfe9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.10 2024-03-04T18:03:48.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.115+0000 7fa00cfe9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 : hit (requested=0x17, cached=0x17) 2024-03-04T18:03:48.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.115+0000 7fa00cfe9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 2024-03-04T18:03:48.111 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.115+0000 7fa00cfe9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:48.111 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.115+0000 7fa00cfe9980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:48.111 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.115+0000 7fa00cfe9980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:48.111 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.115+0000 7fa00cfe9980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17, ofs0, ns] 2024-03-04T18:03:48.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.119+0000 7f9eee7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595101 cookie 94095557991728 notifier 6039 bl.length()=757 2024-03-04T18:03:48.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.119+0000 7f9eee7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:48.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.119+0000 7f9eee7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:48.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.119+0000 7f9eee7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:48.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.119+0000 7fa00cfe9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_sources= new_sources= 2024-03-04T18:03:48.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.119+0000 7fa00cfe9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential sources added=[] removed=[] 2024-03-04T18:03:48.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.119+0000 7fa00cfe9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_dests= new_dests= 2024-03-04T18:03:48.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.119+0000 7fa00cfe9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential dests added=[] removed=[] 2024-03-04T18:03:48.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.127+0000 7fa00cfe9980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-03-04T18:03:48.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.127+0000 7fa00cfe9980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-03-04T18:03:48.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.135+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-03-04T18:03:48.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.135+0000 7fa00cfe9980 20 processing logshard = reshard.0000000007 2024-03-04T18:03:48.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.139+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-03-04T18:03:48.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.139+0000 7fa00cfe9980 20 processing logshard = reshard.0000000008 2024-03-04T18:03:48.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.147+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-03-04T18:03:48.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.147+0000 7fa00cfe9980 20 processing logshard = reshard.0000000009 2024-03-04T18:03:48.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.151+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-03-04T18:03:48.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.151+0000 7fa00cfe9980 20 processing logshard = reshard.0000000010 2024-03-04T18:03:48.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.159+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-03-04T18:03:48.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.159+0000 7fa00cfe9980 20 processing logshard = reshard.0000000011 2024-03-04T18:03:48.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.167+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-03-04T18:03:48.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.167+0000 7fa00cfe9980 20 processing logshard = reshard.0000000012 2024-03-04T18:03:48.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.171+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-03-04T18:03:48.165 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.171+0000 7fa00cfe9980 20 processing logshard = reshard.0000000013 2024-03-04T18:03:48.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.179+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-03-04T18:03:48.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.179+0000 7fa00cfe9980 20 processing logshard = reshard.0000000014 2024-03-04T18:03:48.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.183+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-03-04T18:03:48.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.183+0000 7fa00cfe9980 20 processing logshard = reshard.0000000015 2024-03-04T18:03:48.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.191+0000 7fa00cfe9980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-03-04T18:03:48.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.191+0000 7f9ec17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:48.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.199+0000 7fa00cfe9980 20 remove_watcher() i=0 2024-03-04T18:03:48.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.199+0000 7fa00cfe9980 2 removed watcher, disabling cache 2024-03-04T18:03:48.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.199+0000 7fa00cfe9980 20 remove_watcher() i=1 2024-03-04T18:03:48.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.203+0000 7fa00cfe9980 20 remove_watcher() i=2 2024-03-04T18:03:48.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.207+0000 7fa00cfe9980 20 remove_watcher() i=3 2024-03-04T18:03:48.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.207+0000 7fa00cfe9980 20 remove_watcher() i=4 2024-03-04T18:03:48.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.211+0000 7fa00cfe9980 20 remove_watcher() i=5 2024-03-04T18:03:48.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.215+0000 7fa00cfe9980 20 remove_watcher() i=6 2024-03-04T18:03:48.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:48.215+0000 7fa00cfe9980 20 remove_watcher() i=7 2024-03-04T18:03:53.236 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:03:53.270 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:53.270 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:53.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.502+0000 7f12c00bb640 0 --2- 172.21.15.47:0/3029539068 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55ebb1148060 0x55ebb12c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-03-04T18:03:53.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.526+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:53.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 realm 2024-03-04T18:03:53.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:53.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:53.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:53.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:53.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:53.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.530+0000 7f12c25a3980 20 searching for the correct realm 2024-03-04T18:03:53.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.538+0000 7f12c00bb640 0 -- 172.21.15.47:0/194401631 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f12b00afc20 msgr2=0x7f12b00f4940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:03:53.747 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.750+0000 7f12c00bb640 0 -- 172.21.15.47:0/194401631 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f12b8021060 msgr2=0x7f12b8074810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:53.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:53.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:53.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:53.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:53.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:53.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:53.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:53.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:53.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:53.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 zone default found 2024-03-04T18:03:53.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 4 Realm: () 2024-03-04T18:03:53.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:53.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:53.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:53.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 zonegroup default 2024-03-04T18:03:53.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.974+0000 7f12c25a3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:53.971 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.978+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.978+0000 7f12c25a3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:53.972 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.978+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:53.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.978+0000 7f12c25a3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:53.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.978+0000 7f12c25a3980 20 started sync module instance, tier type = 2024-03-04T18:03:53.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:53.978+0000 7f12c25a3980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:53.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.002+0000 7f12c25a3980 20 add_watcher() i=0 2024-03-04T18:03:53.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.002+0000 7f12c25a3980 20 add_watcher() i=1 2024-03-04T18:03:53.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.002+0000 7f12c25a3980 20 add_watcher() i=2 2024-03-04T18:03:53.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.002+0000 7f12c25a3980 20 add_watcher() i=3 2024-03-04T18:03:53.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.002+0000 7f12c25a3980 20 add_watcher() i=4 2024-03-04T18:03:53.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.002+0000 7f12c25a3980 20 add_watcher() i=5 2024-03-04T18:03:53.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.002+0000 7f12c25a3980 20 add_watcher() i=6 2024-03-04T18:03:54.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.006+0000 7f12c25a3980 20 add_watcher() i=7 2024-03-04T18:03:54.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.006+0000 7f12c25a3980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:54.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.010+0000 7f1176fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:54.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.010+0000 7f12c25a3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:54.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.010+0000 7f12c25a3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:54.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.010+0000 7f12c25a3980 5 note: GC not initialized 2024-03-04T18:03:54.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.010+0000 7f1175fdb640 20 reqs_thread_entry: start 2024-03-04T18:03:54.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.010+0000 7f12c25a3980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:54.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.010+0000 7f12c25a3980 20 Filter name: none 2024-03-04T18:03:54.007 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.010+0000 7f10e77fe640 20 reqs_thread_entry: start 2024-03-04T18:03:54.007 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.010+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.014+0000 7f12c25a3980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:54.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.014+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.014+0000 7f12c25a3980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:54.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.014+0000 7f12c25a3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:54.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.014+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.014+0000 7f12c25a3980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:54.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.014+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.014+0000 7f12c25a3980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:54.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.014+0000 7f12c25a3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:54.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.018+0000 7f12c25a3980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.018+0000 7f12c25a3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:54.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.022+0000 7f1176fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:54.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.026+0000 7f12c25a3980 20 remove_watcher() i=0 2024-03-04T18:03:54.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.026+0000 7f12c25a3980 2 removed watcher, disabling cache 2024-03-04T18:03:54.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.026+0000 7f12c25a3980 20 remove_watcher() i=1 2024-03-04T18:03:54.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.030+0000 7f12c25a3980 20 remove_watcher() i=2 2024-03-04T18:03:54.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.030+0000 7f12c25a3980 20 remove_watcher() i=3 2024-03-04T18:03:54.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.034+0000 7f12c25a3980 20 remove_watcher() i=4 2024-03-04T18:03:54.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.038+0000 7f12c25a3980 20 remove_watcher() i=5 2024-03-04T18:03:54.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.038+0000 7f12c25a3980 20 remove_watcher() i=6 2024-03-04T18:03:54.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.042+0000 7f12c25a3980 20 remove_watcher() i=7 2024-03-04T18:03:54.053 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:54.053 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:03:54.053 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: a-bucket id: 797b67a4-b876-4d04-b122-10afd858d465.4453.17 num_objects: 0 size_kb: 0 num_shards: 12 2024-03-04T18:03:54.053 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:54.053 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard add non empty bucket 2024-03-04T18:03:54.053 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:54.057 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:54.057 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:54.102 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:54.102 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:54.149 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:54.150 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:54.212 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:54.212 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:54.271 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:54.271 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:54.323 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:54.324 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:54.408 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:54.408 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:54.492 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:54.492 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:54.564 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:03:54.597 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:54.597 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:54.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.858+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.858+0000 7f56bbb16980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:54.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.858+0000 7f56bbb16980 20 realm 2024-03-04T18:03:54.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.858+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.862+0000 7f56bbb16980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:54.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.862+0000 7f56bbb16980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:54.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.862+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.862+0000 7f56bbb16980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:54.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.862+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.862+0000 7f56bbb16980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:54.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.862+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:54.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.862+0000 7f56bbb16980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:54.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.862+0000 7f56bbb16980 20 searching for the correct realm 2024-03-04T18:03:54.864 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:54.870+0000 7f56b8e2d640 0 -- 172.21.15.47:0/237095343 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f56b00f4030 msgr2=0x7f56b01067d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:03:55.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.086+0000 7f56b8e2d640 0 -- 172.21.15.47:0/237095343 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x564c523127c0 msgr2=0x564c52314c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:55.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:55.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.098+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 zone default found 2024-03-04T18:03:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 4 Realm: () 2024-03-04T18:03:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 zonegroup default 2024-03-04T18:03:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 started sync module instance, tier type = 2024-03-04T18:03:55.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.102+0000 7f56bbb16980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:55.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.126+0000 7f56bbb16980 20 add_watcher() i=0 2024-03-04T18:03:55.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.126+0000 7f56bbb16980 20 add_watcher() i=1 2024-03-04T18:03:55.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.126+0000 7f56bbb16980 20 add_watcher() i=2 2024-03-04T18:03:55.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.126+0000 7f56bbb16980 20 add_watcher() i=3 2024-03-04T18:03:55.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.126+0000 7f56bbb16980 20 add_watcher() i=4 2024-03-04T18:03:55.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.126+0000 7f56bbb16980 20 add_watcher() i=5 2024-03-04T18:03:55.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.126+0000 7f56bbb16980 20 add_watcher() i=6 2024-03-04T18:03:55.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.130+0000 7f56bbb16980 20 add_watcher() i=7 2024-03-04T18:03:55.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.130+0000 7f56bbb16980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:55.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.134+0000 7f556ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:55.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.134+0000 7f56bbb16980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:55.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.134+0000 7f56bbb16980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:55.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.134+0000 7f56bbb16980 5 note: GC not initialized 2024-03-04T18:03:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.134+0000 7f556efe5640 20 reqs_thread_entry: start 2024-03-04T18:03:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.134+0000 7f56bbb16980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.134+0000 7f56bbb16980 20 Filter name: none 2024-03-04T18:03:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f556cfe1640 20 reqs_thread_entry: start 2024-03-04T18:03:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:55.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:55.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:55.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:55.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:55.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.138+0000 7f56bbb16980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:55.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.142+0000 7f56bbb16980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.142+0000 7f56bbb16980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.142+0000 7f556ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:55.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.150+0000 7f56bbb16980 20 remove_watcher() i=0 2024-03-04T18:03:55.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.150+0000 7f56bbb16980 2 removed watcher, disabling cache 2024-03-04T18:03:55.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.150+0000 7f56bbb16980 20 remove_watcher() i=1 2024-03-04T18:03:55.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.154+0000 7f56bbb16980 20 remove_watcher() i=2 2024-03-04T18:03:55.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.154+0000 7f56bbb16980 20 remove_watcher() i=3 2024-03-04T18:03:55.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.158+0000 7f56bbb16980 20 remove_watcher() i=4 2024-03-04T18:03:55.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.158+0000 7f56bbb16980 20 remove_watcher() i=5 2024-03-04T18:03:55.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.162+0000 7f56bbb16980 20 remove_watcher() i=6 2024-03-04T18:03:55.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.166+0000 7f56bbb16980 20 remove_watcher() i=7 2024-03-04T18:03:55.177 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:55.177 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:03:55.177 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: a-bucket id: 797b67a4-b876-4d04-b122-10afd858d465.4453.17 num_objects: 8 size_kb: 1 num_shards: 12 2024-03-04T18:03:55.177 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:55.177 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 13 2024-03-04T18:03:55.212 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:55.212 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:55.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.262+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 realm 2024-03-04T18:03:55.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:55.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:55.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.262 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:55.262 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.266+0000 7fd613243980 20 searching for the correct realm 2024-03-04T18:03:55.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.274+0000 7fd61055a640 0 -- 172.21.15.47:0/2340800176 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fd5f8043070 msgr2=0x7fd5f8071280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:55.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.490+0000 7fd61155c640 0 -- 172.21.15.47:0/2340800176 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x5647b14b04e0 msgr2=0x5647b14b2970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:55.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:55.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:55.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:55.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:55.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:55.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:55.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:55.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:55.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 zone default found 2024-03-04T18:03:55.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 4 Realm: () 2024-03-04T18:03:55.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:55.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:55.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:55.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 zonegroup default 2024-03-04T18:03:55.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.502+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.506+0000 7fd613243980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.506+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.506+0000 7fd613243980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.506+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.506+0000 7fd613243980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.506+0000 7fd613243980 20 started sync module instance, tier type = 2024-03-04T18:03:55.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.506+0000 7fd613243980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:55.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.526+0000 7fd613243980 20 add_watcher() i=0 2024-03-04T18:03:55.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.526+0000 7fd613243980 20 add_watcher() i=1 2024-03-04T18:03:55.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.526+0000 7fd613243980 20 add_watcher() i=2 2024-03-04T18:03:55.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.526+0000 7fd613243980 20 add_watcher() i=3 2024-03-04T18:03:55.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.526+0000 7fd613243980 20 add_watcher() i=4 2024-03-04T18:03:55.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.526+0000 7fd613243980 20 add_watcher() i=5 2024-03-04T18:03:55.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.526+0000 7fd613243980 20 add_watcher() i=6 2024-03-04T18:03:55.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.530+0000 7fd613243980 20 add_watcher() i=7 2024-03-04T18:03:55.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.530+0000 7fd613243980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:55.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.534+0000 7fd4c6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:55.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.534+0000 7fd613243980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:55.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.534+0000 7fd613243980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:55.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.534+0000 7fd613243980 5 note: GC not initialized 2024-03-04T18:03:55.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.534+0000 7fd4c5fe3640 20 reqs_thread_entry: start 2024-03-04T18:03:55.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:55.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 20 Filter name: none 2024-03-04T18:03:55.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd4377fe640 20 reqs_thread_entry: start 2024-03-04T18:03:55.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:03:55.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:55.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-03-04T18:03:55.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:03:55.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 : miss 2024-03-04T18:03:55.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:55.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:55.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:55.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:55.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:55.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.538+0000 7fd613243980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 2024-03-04T18:03:55.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.542+0000 7fd4c6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:55.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.546+0000 7fd613243980 20 remove_watcher() i=0 2024-03-04T18:03:55.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.546+0000 7fd613243980 2 removed watcher, disabling cache 2024-03-04T18:03:55.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.550+0000 7fd613243980 20 remove_watcher() i=1 2024-03-04T18:03:55.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.550+0000 7fd613243980 20 remove_watcher() i=2 2024-03-04T18:03:55.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.554+0000 7fd613243980 20 remove_watcher() i=3 2024-03-04T18:03:55.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.554+0000 7fd613243980 20 remove_watcher() i=4 2024-03-04T18:03:55.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.558+0000 7fd613243980 20 remove_watcher() i=5 2024-03-04T18:03:55.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.562+0000 7fd613243980 20 remove_watcher() i=6 2024-03-04T18:03:55.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.562+0000 7fd613243980 20 remove_watcher() i=7 2024-03-04T18:03:55.575 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin reshard list 2024-03-04T18:03:55.604 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:55.604 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:55.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.658+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 20 realm 2024-03-04T18:03:55.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:55.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:55.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.662+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.666+0000 7fc569819980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:55.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.666+0000 7fc569819980 20 searching for the correct realm 2024-03-04T18:03:55.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.674+0000 7fc567b32640 0 -- 172.21.15.47:0/1231292460 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x563bbe3d9fb0 msgr2=0x563bbe3dc420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:55.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.678+0000 7fc567331640 0 -- 172.21.15.47:0/1231292460 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x563bbe3de1f0 msgr2=0x563bbe3e05e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:55.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:55.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:55.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:55.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:55.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:55.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:55.681 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 zone default found 2024-03-04T18:03:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 4 Realm: () 2024-03-04T18:03:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:55.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:55.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:55.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 zonegroup default 2024-03-04T18:03:55.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.686+0000 7fc569819980 20 rados->read ofs=0 len=0 2024-03-04T18:03:55.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.690+0000 7fc569819980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:55.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.690+0000 7fc569819980 20 started sync module instance, tier type = 2024-03-04T18:03:55.684 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.690+0000 7fc569819980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:55.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.706+0000 7fc569819980 20 add_watcher() i=0 2024-03-04T18:03:55.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.706+0000 7fc569819980 20 add_watcher() i=1 2024-03-04T18:03:55.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.706+0000 7fc569819980 20 add_watcher() i=2 2024-03-04T18:03:55.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.706+0000 7fc569819980 20 add_watcher() i=3 2024-03-04T18:03:55.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.706+0000 7fc569819980 20 add_watcher() i=4 2024-03-04T18:03:55.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.706+0000 7fc569819980 20 add_watcher() i=5 2024-03-04T18:03:55.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.706+0000 7fc569819980 20 add_watcher() i=6 2024-03-04T18:03:55.705 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.710+0000 7fc569819980 20 add_watcher() i=7 2024-03-04T18:03:55.705 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.710+0000 7fc569819980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:55.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.714+0000 7fc41dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:55.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.714+0000 7fc569819980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:55.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.714+0000 7fc569819980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:55.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.714+0000 7fc569819980 5 note: GC not initialized 2024-03-04T18:03:55.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.714+0000 7fc41cfe1640 20 reqs_thread_entry: start 2024-03-04T18:03:55.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.714+0000 7fc569819980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:55.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.714+0000 7fc569819980 20 Filter name: none 2024-03-04T18:03:55.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.714+0000 7fc3967fc640 20 reqs_thread_entry: start 2024-03-04T18:03:55.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.726+0000 7fc41dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:55.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.730+0000 7fc569819980 20 remove_watcher() i=0 2024-03-04T18:03:55.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.730+0000 7fc569819980 2 removed watcher, disabling cache 2024-03-04T18:03:55.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.730+0000 7fc569819980 20 remove_watcher() i=1 2024-03-04T18:03:55.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.734+0000 7fc569819980 20 remove_watcher() i=2 2024-03-04T18:03:55.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.734+0000 7fc569819980 20 remove_watcher() i=3 2024-03-04T18:03:55.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.738+0000 7fc569819980 20 remove_watcher() i=4 2024-03-04T18:03:55.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.738+0000 7fc569819980 20 remove_watcher() i=5 2024-03-04T18:03:55.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.742+0000 7fc569819980 20 remove_watcher() i=6 2024-03-04T18:03:55.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:55.746+0000 7fc569819980 20 remove_watcher() i=7 2024-03-04T18:03:55.753 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard process non empty bucket 2024-03-04T18:03:55.753 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:55.753 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin reshard process 2024-03-04T18:03:55.776 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:55.776 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:56.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.038+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.038+0000 7f793efa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.038+0000 7f793efa9980 20 realm 2024-03-04T18:03:56.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.038+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.038+0000 7f793efa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.038+0000 7f793efa9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:56.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.038+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.038+0000 7f793efa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.038+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.042+0000 7f793efa9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:56.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.042+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.042+0000 7f793efa9980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:56.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.042+0000 7f793efa9980 20 searching for the correct realm 2024-03-04T18:03:56.041 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.046+0000 7f7937fff640 0 -- 172.21.15.47:0/1523384548 >> [v2:172.21.15.152:6808/929675582,v1:172.21.15.152:6809/929675582] conn(0x55e47c6416f0 msgr2=0x55e47c643b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:56.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.054+0000 7f793cac1640 0 -- 172.21.15.47:0/1523384548 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x55e47c3ff3d0 msgr2=0x7f7938002600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:03:56.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:56.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:56.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:56.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:56.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:56.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:56.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:56.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:56.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 zone default found 2024-03-04T18:03:56.270 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 4 Realm: () 2024-03-04T18:03:56.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:56.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:56.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:56.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 zonegroup default 2024-03-04T18:03:56.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.274+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.271 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.278+0000 7f793efa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.278+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.278+0000 7f793efa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.278+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.278+0000 7f793efa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.278+0000 7f793efa9980 20 started sync module instance, tier type = 2024-03-04T18:03:56.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.278+0000 7f793efa9980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:56.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.302+0000 7f793efa9980 20 add_watcher() i=0 2024-03-04T18:03:56.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.302+0000 7f793efa9980 20 add_watcher() i=1 2024-03-04T18:03:56.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.302+0000 7f793efa9980 20 add_watcher() i=2 2024-03-04T18:03:56.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.302+0000 7f793efa9980 20 add_watcher() i=3 2024-03-04T18:03:56.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.302+0000 7f793efa9980 20 add_watcher() i=4 2024-03-04T18:03:56.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.302+0000 7f793efa9980 20 add_watcher() i=5 2024-03-04T18:03:56.298 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.302+0000 7f793efa9980 20 add_watcher() i=6 2024-03-04T18:03:56.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.306+0000 7f793efa9980 20 add_watcher() i=7 2024-03-04T18:03:56.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.306+0000 7f793efa9980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:56.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.310+0000 7f77f37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:56.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.310+0000 7f793efa9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:56.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.310+0000 7f793efa9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:56.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.310+0000 7f793efa9980 5 note: GC not initialized 2024-03-04T18:03:56.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.310+0000 7f77f27e4640 20 reqs_thread_entry: start 2024-03-04T18:03:56.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.310+0000 7f793efa9980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:56.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.310+0000 7f793efa9980 20 Filter name: none 2024-03-04T18:03:56.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.314+0000 7f77eb7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:56.307 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.314+0000 7f793efa9980 20 processing logshard = reshard.0000000000 2024-03-04T18:03:56.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.318+0000 7f793efa9980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-03-04T18:03:56.314 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.318+0000 7f793efa9980 20 processing logshard = reshard.0000000001 2024-03-04T18:03:56.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.326+0000 7f793efa9980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-03-04T18:03:56.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.326+0000 7f793efa9980 20 processing logshard = reshard.0000000002 2024-03-04T18:03:56.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.334+0000 7f793efa9980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-03-04T18:03:56.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.334+0000 7f793efa9980 20 processing logshard = reshard.0000000003 2024-03-04T18:03:56.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.338+0000 7f793efa9980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-03-04T18:03:56.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.338+0000 7f793efa9980 20 processing logshard = reshard.0000000004 2024-03-04T18:03:56.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.346+0000 7f793efa9980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-03-04T18:03:56.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.346+0000 7f793efa9980 20 processing logshard = reshard.0000000005 2024-03-04T18:03:56.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.350+0000 7f793efa9980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-03-04T18:03:56.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.350+0000 7f793efa9980 20 processing logshard = reshard.0000000006 2024-03-04T18:03:56.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 20 process_entry resharding a-bucket 2024-03-04T18:03:56.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:03:56.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:56.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-03-04T18:03:56.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:03:56.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 : miss 2024-03-04T18:03:56.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:56.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:56.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:56.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:56.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.354+0000 7f793efa9980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:56.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.358+0000 7f793efa9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 2024-03-04T18:03:56.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f793efa9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:56.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f793efa9980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:56.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f793efa9980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:56.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f793efa9980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17, ofs0, ns] 2024-03-04T18:03:56.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f7813fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595102 cookie 94439827853312 notifier 6123 bl.length()=789 2024-03-04T18:03:56.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f7813fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:56.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f7813fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:56.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f7813fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:56.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f793efa9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_sources= new_sources= 2024-03-04T18:03:56.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f793efa9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential sources added=[] removed=[] 2024-03-04T18:03:56.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f793efa9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_dests= new_dests= 2024-03-04T18:03:56.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.374+0000 7f793efa9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential dests added=[] removed=[] 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.0 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.0 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.1 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.1 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.2 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.2 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.3 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.3 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.4 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.4 2024-03-04T18:03:56.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.5 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.5 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.6 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.6 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.7 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.7 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.8 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.8 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.9 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.9 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.10 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.10 2024-03-04T18:03:56.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.11 2024-03-04T18:03:56.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.11 2024-03-04T18:03:56.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.12 2024-03-04T18:03:56.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.12 2024-03-04T18:03:56.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.0 2024-03-04T18:03:56.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.0 2024-03-04T18:03:56.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.1 2024-03-04T18:03:56.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.1 2024-03-04T18:03:56.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.2 2024-03-04T18:03:56.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.382+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.2 2024-03-04T18:03:56.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.386+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.3 2024-03-04T18:03:56.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.386+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.3 2024-03-04T18:03:56.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.386+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.4 2024-03-04T18:03:56.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.386+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.4 2024-03-04T18:03:56.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.386+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.5 2024-03-04T18:03:56.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.386+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.5 2024-03-04T18:03:56.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.386+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.6 2024-03-04T18:03:56.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.386+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.6 2024-03-04T18:03:56.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.390+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.7 2024-03-04T18:03:56.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.390+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.7 2024-03-04T18:03:56.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.390+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.8 2024-03-04T18:03:56.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.390+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.8 2024-03-04T18:03:56.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.390+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.9 2024-03-04T18:03:56.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.390+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.9 2024-03-04T18:03:56.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.394+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.10 2024-03-04T18:03:56.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.394+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.10 2024-03-04T18:03:56.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.394+0000 7f793efa9980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.11 2024-03-04T18:03:56.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.394+0000 7f793efa9980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.1.11 2024-03-04T18:03:56.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.398+0000 7f793efa9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 : hit (requested=0x17, cached=0x17) 2024-03-04T18:03:56.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.398+0000 7f793efa9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 2024-03-04T18:03:56.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.402+0000 7f793efa9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:56.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.402+0000 7f793efa9980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:56.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.402+0000 7f793efa9980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:56.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.402+0000 7f793efa9980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17, ofs0, ns] 2024-03-04T18:03:56.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.402+0000 7f78137fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595103 cookie 94439827853312 notifier 6123 bl.length()=757 2024-03-04T18:03:56.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.402+0000 7f78137fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:03:56.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.402+0000 7f78137fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:03:56.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.402+0000 7f78137fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:03:56.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.406+0000 7f793efa9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_sources= new_sources= 2024-03-04T18:03:56.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.406+0000 7f793efa9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential sources added=[] removed=[] 2024-03-04T18:03:56.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.406+0000 7f793efa9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_dests= new_dests= 2024-03-04T18:03:56.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.406+0000 7f793efa9980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential dests added=[] removed=[] 2024-03-04T18:03:56.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.410+0000 7f793efa9980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-03-04T18:03:56.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.414+0000 7f793efa9980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-03-04T18:03:56.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.422+0000 7f793efa9980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-03-04T18:03:56.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.422+0000 7f793efa9980 20 processing logshard = reshard.0000000007 2024-03-04T18:03:56.420 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.426+0000 7f793efa9980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-03-04T18:03:56.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.426+0000 7f793efa9980 20 processing logshard = reshard.0000000008 2024-03-04T18:03:56.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.430+0000 7f793efa9980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-03-04T18:03:56.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.430+0000 7f793efa9980 20 processing logshard = reshard.0000000009 2024-03-04T18:03:56.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.434+0000 7f793efa9980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-03-04T18:03:56.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.434+0000 7f793efa9980 20 processing logshard = reshard.0000000010 2024-03-04T18:03:56.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.442+0000 7f793efa9980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-03-04T18:03:56.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.442+0000 7f793efa9980 20 processing logshard = reshard.0000000011 2024-03-04T18:03:56.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.450+0000 7f793efa9980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-03-04T18:03:56.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.450+0000 7f793efa9980 20 processing logshard = reshard.0000000012 2024-03-04T18:03:56.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.454+0000 7f793efa9980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-03-04T18:03:56.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.454+0000 7f793efa9980 20 processing logshard = reshard.0000000013 2024-03-04T18:03:56.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.458+0000 7f793efa9980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-03-04T18:03:56.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.458+0000 7f793efa9980 20 processing logshard = reshard.0000000014 2024-03-04T18:03:56.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.466+0000 7f793efa9980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-03-04T18:03:56.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.466+0000 7f793efa9980 20 processing logshard = reshard.0000000015 2024-03-04T18:03:56.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.470+0000 7f793efa9980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-03-04T18:03:56.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.474+0000 7f77f37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:56.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.478+0000 7f793efa9980 20 remove_watcher() i=0 2024-03-04T18:03:56.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.478+0000 7f793efa9980 2 removed watcher, disabling cache 2024-03-04T18:03:56.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.482+0000 7f793efa9980 20 remove_watcher() i=1 2024-03-04T18:03:56.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.482+0000 7f793efa9980 20 remove_watcher() i=2 2024-03-04T18:03:56.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.486+0000 7f793efa9980 20 remove_watcher() i=3 2024-03-04T18:03:56.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.486+0000 7f793efa9980 20 remove_watcher() i=4 2024-03-04T18:03:56.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.490+0000 7f793efa9980 20 remove_watcher() i=5 2024-03-04T18:03:56.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.494+0000 7f793efa9980 20 remove_watcher() i=6 2024-03-04T18:03:56.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.494+0000 7f793efa9980 20 remove_watcher() i=7 2024-03-04T18:03:56.510 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:03:56.541 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:56.542 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:56.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.786+0000 7f692f75a640 0 --2- 172.21.15.47:0/1092676702 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55953557c760 0x5595355773e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-03-04T18:03:56.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.798+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.802+0000 7f6932443980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.802+0000 7f6932443980 20 realm 2024-03-04T18:03:56.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.802+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.802+0000 7f6932443980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.802+0000 7f6932443980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:56.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.802+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.802+0000 7f6932443980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:56.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.802+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.806+0000 7f6932443980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:56.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.806+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:56.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.806+0000 7f6932443980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:56.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.806+0000 7f6932443980 20 searching for the correct realm 2024-03-04T18:03:56.806 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:56.810+0000 7f692f75a640 0 -- 172.21.15.47:0/1329181950 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f69240ee450 msgr2=0x7f69240ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:57.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.022+0000 7f693075c640 0 -- 172.21.15.47:0/1329181950 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x5595355e16a0 msgr2=0x5595355e3b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:57.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:57.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:57.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:57.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:57.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:57.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:57.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:57.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:57.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:57.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 zone default found 2024-03-04T18:03:57.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 4 Realm: () 2024-03-04T18:03:57.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:57.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:57.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:57.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 zonegroup default 2024-03-04T18:03:57.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.038+0000 7f6932443980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:57.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.042+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.042+0000 7f6932443980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:57.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.042+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.042+0000 7f6932443980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:57.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.042+0000 7f6932443980 20 started sync module instance, tier type = 2024-03-04T18:03:57.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.042+0000 7f6932443980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:57.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.066+0000 7f6932443980 20 add_watcher() i=0 2024-03-04T18:03:57.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.066+0000 7f6932443980 20 add_watcher() i=1 2024-03-04T18:03:57.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.066+0000 7f6932443980 20 add_watcher() i=2 2024-03-04T18:03:57.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.066+0000 7f6932443980 20 add_watcher() i=3 2024-03-04T18:03:57.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.066+0000 7f6932443980 20 add_watcher() i=4 2024-03-04T18:03:57.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.066+0000 7f6932443980 20 add_watcher() i=5 2024-03-04T18:03:57.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.066+0000 7f6932443980 20 add_watcher() i=6 2024-03-04T18:03:57.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.070+0000 7f6932443980 20 add_watcher() i=7 2024-03-04T18:03:57.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.070+0000 7f6932443980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:57.067 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f67e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:57.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f6932443980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:57.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f6932443980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:57.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f6932443980 5 note: GC not initialized 2024-03-04T18:03:57.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f67e4fe1640 20 reqs_thread_entry: start 2024-03-04T18:03:57.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f6932443980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:57.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f6932443980 20 Filter name: none 2024-03-04T18:03:57.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f675e7fc640 20 reqs_thread_entry: start 2024-03-04T18:03:57.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f6932443980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:57.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.074+0000 7f6932443980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:57.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.078+0000 7f6932443980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:57.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.078+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.078+0000 7f6932443980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:03:57.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.078+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.078+0000 7f6932443980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:03:57.072 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.078+0000 7f6932443980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:57.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.078+0000 7f6932443980 20 rados->read ofs=0 len=0 2024-03-04T18:03:57.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.078+0000 7f6932443980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:57.076 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.082+0000 7f67e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:57.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.086+0000 7f6932443980 20 remove_watcher() i=0 2024-03-04T18:03:57.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.086+0000 7f6932443980 2 removed watcher, disabling cache 2024-03-04T18:03:57.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.086+0000 7f6932443980 20 remove_watcher() i=1 2024-03-04T18:03:57.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.090+0000 7f6932443980 20 remove_watcher() i=2 2024-03-04T18:03:57.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.094+0000 7f6932443980 20 remove_watcher() i=3 2024-03-04T18:03:57.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.094+0000 7f6932443980 20 remove_watcher() i=4 2024-03-04T18:03:57.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.098+0000 7f6932443980 20 remove_watcher() i=5 2024-03-04T18:03:57.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.098+0000 7f6932443980 20 remove_watcher() i=6 2024-03-04T18:03:57.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:57.102+0000 7f6932443980 20 remove_watcher() i=7 2024-03-04T18:03:57.113 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:57.113 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:03:57.113 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: a-bucket id: 797b67a4-b876-4d04-b122-10afd858d465.4453.17 num_objects: 8 size_kb: 1 num_shards: 13 2024-03-04T18:03:57.113 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:57.113 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with EIO injected at set_target_layout 2024-03-04T18:03:57.113 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:57.114 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.114 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.201 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.201 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.295 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.296 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.384 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.384 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.468 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.468 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.552 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.552 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.613 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.613 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.695 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.695 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.781 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.781 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.865 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.865 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:57.945 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:57.945 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.031 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.032 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.119 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.119 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.202 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.203 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.286 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.286 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.369 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.369 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.448 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.449 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.532 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.532 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.591 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.591 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.677 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.677 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.761 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.761 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.841 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:58.841 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:03:58.926 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:03:58.961 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:58.961 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:59.007 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.014+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.014+0000 7fa550b1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.014+0000 7fa550b1c980 20 realm 2024-03-04T18:03:59.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.014+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.014+0000 7fa550b1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.014+0000 7fa550b1c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:59.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.014+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.018+0000 7fa550b1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.018+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.018+0000 7fa550b1c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:59.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.018+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.018+0000 7fa550b1c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:59.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.018+0000 7fa550b1c980 20 searching for the correct realm 2024-03-04T18:03:59.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.022+0000 7fa54e634640 0 -- 172.21.15.47:0/1742499233 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fa548015030 msgr2=0x7fa54803f420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:59.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.234+0000 7fa54e634640 0 -- 172.21.15.47:0/1742499233 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fa538003ac0 msgr2=0x7fa54007a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:59.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:59.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:59.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:59.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:59.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:59.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:59.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:59.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.454+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:59.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 zone default found 2024-03-04T18:03:59.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 4 Realm: () 2024-03-04T18:03:59.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:59.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:59.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:59.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 zonegroup default 2024-03-04T18:03:59.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 started sync module instance, tier type = 2024-03-04T18:03:59.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.458+0000 7fa550b1c980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:59.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.482+0000 7fa550b1c980 20 add_watcher() i=0 2024-03-04T18:03:59.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.482+0000 7fa550b1c980 20 add_watcher() i=1 2024-03-04T18:03:59.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.482+0000 7fa550b1c980 20 add_watcher() i=2 2024-03-04T18:03:59.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.482+0000 7fa550b1c980 20 add_watcher() i=3 2024-03-04T18:03:59.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.482+0000 7fa550b1c980 20 add_watcher() i=4 2024-03-04T18:03:59.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.482+0000 7fa550b1c980 20 add_watcher() i=5 2024-03-04T18:03:59.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.482+0000 7fa550b1c980 20 add_watcher() i=6 2024-03-04T18:03:59.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.486+0000 7fa550b1c980 20 add_watcher() i=7 2024-03-04T18:03:59.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.486+0000 7fa550b1c980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.490+0000 7fa404fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.490+0000 7fa550b1c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.490+0000 7fa550b1c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.490+0000 7fa550b1c980 5 note: GC not initialized 2024-03-04T18:03:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.490+0000 7fa37f7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.494+0000 7fa550b1c980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.494+0000 7fa550b1c980 20 Filter name: none 2024-03-04T18:03:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.494+0000 7fa37d7fa640 20 reqs_thread_entry: start 2024-03-04T18:03:59.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.494+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.494+0000 7fa550b1c980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:03:59.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.494+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.494+0000 7fa550b1c980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:03:59.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.494+0000 7fa550b1c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:59.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.494+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.498+0000 7fa550b1c980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:03:59.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.498+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.498+0000 7fa550b1c980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:03:59.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.498+0000 7fa550b1c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:59.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.498+0000 7fa550b1c980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.502+0000 7fa550b1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.502+0000 7fa404fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:59.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.506+0000 7fa550b1c980 20 remove_watcher() i=0 2024-03-04T18:03:59.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.506+0000 7fa550b1c980 2 removed watcher, disabling cache 2024-03-04T18:03:59.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.506+0000 7fa550b1c980 20 remove_watcher() i=1 2024-03-04T18:03:59.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.510+0000 7fa550b1c980 20 remove_watcher() i=2 2024-03-04T18:03:59.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.514+0000 7fa550b1c980 20 remove_watcher() i=3 2024-03-04T18:03:59.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.514+0000 7fa550b1c980 20 remove_watcher() i=4 2024-03-04T18:03:59.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.518+0000 7fa550b1c980 20 remove_watcher() i=5 2024-03-04T18:03:59.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.522+0000 7fa550b1c980 20 remove_watcher() i=6 2024-03-04T18:03:59.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.522+0000 7fa550b1c980 20 remove_watcher() i=7 2024-03-04T18:03:59.534 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:59.534 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:03:59.534 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-set-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.19 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:03:59.534 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:03:59.534 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout 2024-03-04T18:03:59.572 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:59.572 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:59.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.618+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 realm 2024-03-04T18:03:59.616 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:59.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.617 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:59.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:59.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.622+0000 7f29304f4980 20 searching for the correct realm 2024-03-04T18:03:59.628 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.634+0000 7f292e80d640 0 -- 172.21.15.47:0/174052684 >> [v2:172.21.15.152:6818/159740591,v1:172.21.15.152:6821/159740591] conn(0x7f29200ccc40 msgr2=0x7f292011f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:03:59.845 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.850+0000 7f292d80b640 0 -- 172.21.15.47:0/174052684 >> [v2:172.21.15.152:6808/929675582,v1:172.21.15.152:6809/929675582] conn(0x560c41a1afb0 msgr2=0x560c41a1d440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:59.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.854+0000 7f29304f4980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:03:59.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.854+0000 7f29304f4980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:03:59.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.854+0000 7f29304f4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:59.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.854+0000 7f29304f4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:59.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.854+0000 7f29304f4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:59.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.854+0000 7f29304f4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:59.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.854+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.854+0000 7f29304f4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.854+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:59.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:59.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 zone default found 2024-03-04T18:03:59.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 4 Realm: () 2024-03-04T18:03:59.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:03:59.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:03:59.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:59.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 zonegroup default 2024-03-04T18:03:59.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.858+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.862+0000 7f29304f4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:59.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.862+0000 7f29304f4980 20 started sync module instance, tier type = 2024-03-04T18:03:59.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.862+0000 7f29304f4980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:03:59.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.882+0000 7f29304f4980 20 add_watcher() i=0 2024-03-04T18:03:59.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.886+0000 7f29304f4980 20 add_watcher() i=1 2024-03-04T18:03:59.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.886+0000 7f29304f4980 20 add_watcher() i=2 2024-03-04T18:03:59.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.886+0000 7f29304f4980 20 add_watcher() i=3 2024-03-04T18:03:59.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.886+0000 7f29304f4980 20 add_watcher() i=4 2024-03-04T18:03:59.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.886+0000 7f29304f4980 20 add_watcher() i=5 2024-03-04T18:03:59.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.886+0000 7f29304f4980 20 add_watcher() i=6 2024-03-04T18:03:59.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.886+0000 7f29304f4980 20 add_watcher() i=7 2024-03-04T18:03:59.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.886+0000 7f29304f4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:59.886 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.890+0000 7f27e4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:59.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.890+0000 7f29304f4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:59.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.890+0000 7f29304f4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:59.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.890+0000 7f29304f4980 5 note: GC not initialized 2024-03-04T18:03:59.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f275f7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:59.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 20 init_complete bucket index max shards: 11 2024-03-04T18:03:59.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 20 Filter name: none 2024-03-04T18:03:59.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f275d7fa640 20 reqs_thread_entry: start 2024-03-04T18:03:59.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-03-04T18:03:59.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:03:59.889 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-03-04T18:03:59.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-03-04T18:03:59.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 : miss 2024-03-04T18:03:59.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:59.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:03:59.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 info.flags=0x17 2024-03-04T18:03:59.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 to cache LRU end 2024-03-04T18:03:59.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:59.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:03:59.891 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.894+0000 7f29304f4980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 2024-03-04T18:03:59.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.902+0000 7f29304f4980 -1 Injecting error=-5 at location=set_target_layout 2024-03-04T18:03:59.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.902+0000 7f29304f4980 0 ERROR: init_target_layout failed to write target index layout to bucket info: (5) Input/output error 2024-03-04T18:03:59.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.914+0000 7f27e4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:59.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.918+0000 7f29304f4980 20 remove_watcher() i=0 2024-03-04T18:03:59.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.918+0000 7f29304f4980 2 removed watcher, disabling cache 2024-03-04T18:03:59.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.918+0000 7f29304f4980 20 remove_watcher() i=1 2024-03-04T18:03:59.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.922+0000 7f29304f4980 20 remove_watcher() i=2 2024-03-04T18:03:59.919 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.926+0000 7f29304f4980 20 remove_watcher() i=3 2024-03-04T18:03:59.921 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.926+0000 7f29304f4980 20 remove_watcher() i=4 2024-03-04T18:03:59.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.930+0000 7f29304f4980 20 remove_watcher() i=5 2024-03-04T18:03:59.927 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.930+0000 7f29304f4980 20 remove_watcher() i=6 2024-03-04T18:03:59.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:03:59.934+0000 7f29304f4980 20 remove_watcher() i=7 2024-03-04T18:03:59.946 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:03:59.974 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:59.974 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:00.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.026+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.026+0000 7f352d1dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.026+0000 7f352d1dd980 20 realm 2024-03-04T18:04:00.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.026+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.026+0000 7f352d1dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.026+0000 7f352d1dd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:00.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.026+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.026+0000 7f352d1dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.026+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.030+0000 7f352d1dd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:00.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.030+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.030+0000 7f352d1dd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:00.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.030+0000 7f352d1dd980 20 searching for the correct realm 2024-03-04T18:04:00.028 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.034+0000 7f352acf5640 0 -- 172.21.15.47:0/309760153 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f351c010040 msgr2=0x7f351c07ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:00.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.246+0000 7f352acf5640 0 -- 172.21.15.47:0/309760153 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x55a3c28ea060 msgr2=0x7f3524034410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:00.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:00.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:00.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:00.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:00.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:00.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:00.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:00.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:00.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 zone default found 2024-03-04T18:04:00.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 4 Realm: () 2024-03-04T18:04:00.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:00.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:00.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:00.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 zonegroup default 2024-03-04T18:04:00.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.466+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.470+0000 7f352d1dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.470+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.470+0000 7f352d1dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.470+0000 7f352d1dd980 20 started sync module instance, tier type = 2024-03-04T18:04:00.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.470+0000 7f352d1dd980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:00.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.494+0000 7f352d1dd980 20 add_watcher() i=0 2024-03-04T18:04:00.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.494+0000 7f352d1dd980 20 add_watcher() i=1 2024-03-04T18:04:00.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.494+0000 7f352d1dd980 20 add_watcher() i=2 2024-03-04T18:04:00.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.494+0000 7f352d1dd980 20 add_watcher() i=3 2024-03-04T18:04:00.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.494+0000 7f352d1dd980 20 add_watcher() i=4 2024-03-04T18:04:00.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.494+0000 7f352d1dd980 20 add_watcher() i=5 2024-03-04T18:04:00.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.494+0000 7f352d1dd980 20 add_watcher() i=6 2024-03-04T18:04:00.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.494+0000 7f352d1dd980 20 add_watcher() i=7 2024-03-04T18:04:00.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.494+0000 7f352d1dd980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:00.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.498+0000 7f33e17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:00.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:00.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:00.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 5 note: GC not initialized 2024-03-04T18:04:00.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f335bfff640 20 reqs_thread_entry: start 2024-03-04T18:04:00.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:00.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 Filter name: none 2024-03-04T18:04:00.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f3359ffb640 20 reqs_thread_entry: start 2024-03-04T18:04:00.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:00.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:00.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:00.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.502+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.506+0000 7f352d1dd980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:00.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.506+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.506+0000 7f352d1dd980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:00.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.506+0000 7f352d1dd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:00.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.510+0000 7f352d1dd980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.510+0000 7f352d1dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.510+0000 7f33e17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:00.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.514+0000 7f352d1dd980 20 remove_watcher() i=0 2024-03-04T18:04:00.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.514+0000 7f352d1dd980 2 removed watcher, disabling cache 2024-03-04T18:04:00.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.518+0000 7f352d1dd980 20 remove_watcher() i=1 2024-03-04T18:04:00.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.518+0000 7f352d1dd980 20 remove_watcher() i=2 2024-03-04T18:04:00.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.522+0000 7f352d1dd980 20 remove_watcher() i=3 2024-03-04T18:04:00.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.522+0000 7f352d1dd980 20 remove_watcher() i=4 2024-03-04T18:04:00.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.526+0000 7f352d1dd980 20 remove_watcher() i=5 2024-03-04T18:04:00.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.530+0000 7f352d1dd980 20 remove_watcher() i=6 2024-03-04T18:04:00.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.530+0000 7f352d1dd980 20 remove_watcher() i=7 2024-03-04T18:04:00.544 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:00.544 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:00.544 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-set-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.19 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:04:00.544 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:00.546 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:00.546 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:00.595 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:00.595 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:00.669 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:00.705 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:00.705 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:00.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.754+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 realm 2024-03-04T18:04:00.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:00.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:00.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.754 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:00.754 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.758+0000 7f3e87d30980 20 searching for the correct realm 2024-03-04T18:04:00.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.766+0000 7f3e86049640 0 -- 172.21.15.47:0/1914044576 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f3e7c001000 msgr2=0x7f3e7c0b73c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:00.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.974+0000 7f3e86049640 0 -- 172.21.15.47:0/1914044576 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x559e6e90d0b0 msgr2=0x559e6e90f4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:00.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:00.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:00.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:00.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:00.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:00.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:00.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:00.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:00.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 zone default found 2024-03-04T18:04:00.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 4 Realm: () 2024-03-04T18:04:00.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:00.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:00.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:00.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 zonegroup default 2024-03-04T18:04:00.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.990+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.994+0000 7f3e87d30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.994+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:00.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.994+0000 7f3e87d30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:00.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.994+0000 7f3e87d30980 20 started sync module instance, tier type = 2024-03-04T18:04:00.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:00.994+0000 7f3e87d30980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:01.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.018+0000 7f3e87d30980 20 add_watcher() i=0 2024-03-04T18:04:01.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.018+0000 7f3e87d30980 20 add_watcher() i=1 2024-03-04T18:04:01.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.018+0000 7f3e87d30980 20 add_watcher() i=2 2024-03-04T18:04:01.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.018+0000 7f3e87d30980 20 add_watcher() i=3 2024-03-04T18:04:01.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.018+0000 7f3e87d30980 20 add_watcher() i=4 2024-03-04T18:04:01.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.018+0000 7f3e87d30980 20 add_watcher() i=5 2024-03-04T18:04:01.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.018+0000 7f3e87d30980 20 add_watcher() i=6 2024-03-04T18:04:01.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.018+0000 7f3e87d30980 20 add_watcher() i=7 2024-03-04T18:04:01.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.018+0000 7f3e87d30980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:01.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3d377fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:01.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:01.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:01.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 5 note: GC not initialized 2024-03-04T18:04:01.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3d367fc640 20 reqs_thread_entry: start 2024-03-04T18:04:01.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:01.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 20 Filter name: none 2024-03-04T18:04:01.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3ca7fff640 20 reqs_thread_entry: start 2024-03-04T18:04:01.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-03-04T18:04:01.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:01.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-03-04T18:04:01.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-03-04T18:04:01.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 : miss 2024-03-04T18:04:01.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.026+0000 7f3e87d30980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.030+0000 7f3e87d30980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:01.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.030+0000 7f3e87d30980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 info.flags=0x17 2024-03-04T18:04:01.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.030+0000 7f3e87d30980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 to cache LRU end 2024-03-04T18:04:01.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.030+0000 7f3e87d30980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:01.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.030+0000 7f3e87d30980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:01.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.030+0000 7f3e87d30980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 2024-03-04T18:04:01.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.038+0000 7f3e87d30980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 info.flags=0x17 2024-03-04T18:04:01.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.038+0000 7f3e87d30980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 to cache LRU end 2024-03-04T18:04:01.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.038+0000 7f3e87d30980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:01.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.038+0000 7f3e87d30980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19, ofs0, ns] 2024-03-04T18:04:01.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.042+0000 7f3d697fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595105 cookie 94138948230288 notifier 5867 bl.length()=887 2024-03-04T18:04:01.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.042+0000 7f3d697fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 info.flags=0x17 2024-03-04T18:04:01.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.042+0000 7f3d697fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 to cache LRU end 2024-03-04T18:04:01.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.042+0000 7f3d697fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:01.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.042+0000 7f3e87d30980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.19]): orig_sources= new_sources= 2024-03-04T18:04:01.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.042+0000 7f3e87d30980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.19]): potential sources added=[] removed=[] 2024-03-04T18:04:01.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.042+0000 7f3e87d30980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.19]): orig_dests= new_dests= 2024-03-04T18:04:01.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.042+0000 7f3e87d30980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.19]): potential dests added=[] removed=[] 2024-03-04T18:04:01.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.0 2024-03-04T18:04:01.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.0 2024-03-04T18:04:01.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.1 2024-03-04T18:04:01.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.1 2024-03-04T18:04:01.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.2 2024-03-04T18:04:01.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.2 2024-03-04T18:04:01.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.3 2024-03-04T18:04:01.043 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.3 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.4 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.4 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.5 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.5 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.6 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.6 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.7 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.7 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.8 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.8 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.9 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.9 2024-03-04T18:04:01.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.10 2024-03-04T18:04:01.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.10 2024-03-04T18:04:01.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.11 2024-03-04T18:04:01.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1.11 2024-03-04T18:04:01.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.046+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.0 2024-03-04T18:04:01.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.050+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.1 2024-03-04T18:04:01.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.050+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.2 2024-03-04T18:04:01.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.050+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.3 2024-03-04T18:04:01.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.050+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.4 2024-03-04T18:04:01.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.054+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.5 2024-03-04T18:04:01.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.054+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.6 2024-03-04T18:04:01.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.054+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.7 2024-03-04T18:04:01.050 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.054+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.8 2024-03-04T18:04:01.051 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.054+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.9 2024-03-04T18:04:01.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.058+0000 7f3e87d30980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.19.10 2024-03-04T18:04:01.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3e87d30980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:01.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3e87d30980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 2024-03-04T18:04:01.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3e87d30980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 info.flags=0x17 2024-03-04T18:04:01.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3e87d30980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 to cache LRU end 2024-03-04T18:04:01.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3e87d30980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:01.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3e87d30980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19, ofs0, ns] 2024-03-04T18:04:01.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3d697fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595106 cookie 94138948230288 notifier 5867 bl.length()=855 2024-03-04T18:04:01.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3d697fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 info.flags=0x17 2024-03-04T18:04:01.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3d697fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.19 to cache LRU end 2024-03-04T18:04:01.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.066+0000 7f3d697fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:01.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.070+0000 7f3e87d30980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.19]): orig_sources= new_sources= 2024-03-04T18:04:01.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.070+0000 7f3e87d30980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.19]): potential sources added=[] removed=[] 2024-03-04T18:04:01.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.070+0000 7f3e87d30980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.19]): orig_dests= new_dests= 2024-03-04T18:04:01.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.070+0000 7f3e87d30980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.19]): potential dests added=[] removed=[] 2024-03-04T18:04:01.071 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.074+0000 7f3e87d30980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-03-04T18:04:01.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.078+0000 7f3d377fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:01.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.082+0000 7f3e87d30980 20 remove_watcher() i=0 2024-03-04T18:04:01.079 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.082+0000 7f3e87d30980 2 removed watcher, disabling cache 2024-03-04T18:04:01.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.086+0000 7f3e87d30980 20 remove_watcher() i=1 2024-03-04T18:04:01.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.090+0000 7f3e87d30980 20 remove_watcher() i=2 2024-03-04T18:04:01.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.090+0000 7f3e87d30980 20 remove_watcher() i=3 2024-03-04T18:04:01.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.094+0000 7f3e87d30980 20 remove_watcher() i=4 2024-03-04T18:04:01.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.094+0000 7f3e87d30980 20 remove_watcher() i=5 2024-03-04T18:04:01.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.098+0000 7f3e87d30980 20 remove_watcher() i=6 2024-03-04T18:04:01.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.102+0000 7f3e87d30980 20 remove_watcher() i=7 2024-03-04T18:04:01.115 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:04:01.146 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:01.146 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:01.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.198+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.198+0000 7f9f26ce8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:01.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.198+0000 7f9f26ce8980 20 realm 2024-03-04T18:04:01.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.198+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.202+0000 7f9f26ce8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:01.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.202+0000 7f9f26ce8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:01.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.202+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.202+0000 7f9f26ce8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:01.196 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.202+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.202+0000 7f9f26ce8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:01.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.202+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.202+0000 7f9f26ce8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:01.198 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.202+0000 7f9f26ce8980 20 searching for the correct realm 2024-03-04T18:04:01.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.210+0000 7f9f25001640 0 -- 172.21.15.47:0/3848652894 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f9f1806ebe0 msgr2=0x7f9f18059b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:01.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.218+0000 7f9f24800640 0 -- 172.21.15.47:0/3848652894 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f9f140f5040 msgr2=0x7f9f141079c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:01.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.222+0000 7f9f26ce8980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:01.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.222+0000 7f9f26ce8980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:01.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.222+0000 7f9f26ce8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:01.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.222+0000 7f9f26ce8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:01.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.222+0000 7f9f26ce8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:01.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.222+0000 7f9f26ce8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:01.219 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.222+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:01.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:01.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:01.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 zone default found 2024-03-04T18:04:01.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 4 Realm: () 2024-03-04T18:04:01.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:01.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:01.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:01.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 zonegroup default 2024-03-04T18:04:01.221 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:01.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:01.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:01.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 started sync module instance, tier type = 2024-03-04T18:04:01.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.226+0000 7f9f26ce8980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:01.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.246+0000 7f9f26ce8980 20 add_watcher() i=0 2024-03-04T18:04:01.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.246+0000 7f9f26ce8980 20 add_watcher() i=1 2024-03-04T18:04:01.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.246+0000 7f9f26ce8980 20 add_watcher() i=2 2024-03-04T18:04:01.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.246+0000 7f9f26ce8980 20 add_watcher() i=3 2024-03-04T18:04:01.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.246+0000 7f9f26ce8980 20 add_watcher() i=4 2024-03-04T18:04:01.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.246+0000 7f9f26ce8980 20 add_watcher() i=5 2024-03-04T18:04:01.240 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.246+0000 7f9f26ce8980 20 add_watcher() i=6 2024-03-04T18:04:01.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.246+0000 7f9f26ce8980 20 add_watcher() i=7 2024-03-04T18:04:01.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.246+0000 7f9f26ce8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:01.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.250+0000 7f9ddafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:01.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.250+0000 7f9f26ce8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:01.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.250+0000 7f9f26ce8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:01.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.250+0000 7f9f26ce8980 5 note: GC not initialized 2024-03-04T18:04:01.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.250+0000 7f9dd9fdb640 20 reqs_thread_entry: start 2024-03-04T18:04:01.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.250+0000 7f9f26ce8980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:01.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.250+0000 7f9f26ce8980 20 Filter name: none 2024-03-04T18:04:01.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.250+0000 7f9d4b7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:01.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.250+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.254+0000 7f9f26ce8980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:01.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.254+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.254+0000 7f9f26ce8980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:01.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.254+0000 7f9f26ce8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:01.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.254+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.254+0000 7f9f26ce8980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:01.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.254+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.258+0000 7f9f26ce8980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:01.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.258+0000 7f9f26ce8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:01.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.258+0000 7f9f26ce8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:01.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.258+0000 7f9f26ce8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:01.254 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.258+0000 7f9ddafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:01.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.266+0000 7f9f26ce8980 20 remove_watcher() i=0 2024-03-04T18:04:01.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.266+0000 7f9f26ce8980 2 removed watcher, disabling cache 2024-03-04T18:04:01.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.266+0000 7f9f26ce8980 20 remove_watcher() i=1 2024-03-04T18:04:01.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.270+0000 7f9f26ce8980 20 remove_watcher() i=2 2024-03-04T18:04:01.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.270+0000 7f9f26ce8980 20 remove_watcher() i=3 2024-03-04T18:04:01.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.274+0000 7f9f26ce8980 20 remove_watcher() i=4 2024-03-04T18:04:01.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.274+0000 7f9f26ce8980 20 remove_watcher() i=5 2024-03-04T18:04:01.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.278+0000 7f9f26ce8980 20 remove_watcher() i=6 2024-03-04T18:04:01.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:01.278+0000 7f9f26ce8980 20 remove_watcher() i=7 2024-03-04T18:04:01.289 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:01.290 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:01.290 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-set-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.19 num_objects: 19 size_kb: 1 num_shards: 12 2024-03-04T18:04:01.290 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:01.291 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:01.291 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:01.313 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:01.313 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:01.726 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:01.726 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.102 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with ECANCELED injected at set_target_layout 2024-03-04T18:04:02.102 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:02.104 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.104 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.232 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.232 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.311 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.311 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.396 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.396 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.484 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.484 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.571 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.571 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.635 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.635 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.715 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.715 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.801 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.801 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.885 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.885 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:02.948 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:02.949 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.032 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.033 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.122 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.122 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.207 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.207 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.292 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.292 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.369 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.369 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.455 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.455 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.542 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.542 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.626 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.627 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.706 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.707 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.791 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.791 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.873 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:03.873 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:03.950 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:04:03.983 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:03.983 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:04.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.034+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 realm 2024-03-04T18:04:04.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:04.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:04.035 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.038+0000 7fcbf7245980 20 searching for the correct realm 2024-03-04T18:04:04.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.042+0000 7fcbf4d5d640 0 -- 172.21.15.47:0/3875637637 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fcbe40ca070 msgr2=0x7fcbe40eea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:04.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.050+0000 7fcbf4d5d640 0 -- 172.21.15.47:0/3875637637 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x559e25c92060 msgr2=0x7fcbec056560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:04.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:04.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:04.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:04.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:04.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:04.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:04.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:04.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 zone default found 2024-03-04T18:04:04.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 4 Realm: () 2024-03-04T18:04:04.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:04.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:04.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:04.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 zonegroup default 2024-03-04T18:04:04.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.270+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.274+0000 7fcbf7245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.274+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.274+0000 7fcbf7245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.274+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.274+0000 7fcbf7245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.274+0000 7fcbf7245980 20 started sync module instance, tier type = 2024-03-04T18:04:04.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.274+0000 7fcbf7245980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:04.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.298+0000 7fcbf7245980 20 add_watcher() i=0 2024-03-04T18:04:04.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.298+0000 7fcbf7245980 20 add_watcher() i=1 2024-03-04T18:04:04.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.298+0000 7fcbf7245980 20 add_watcher() i=2 2024-03-04T18:04:04.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.298+0000 7fcbf7245980 20 add_watcher() i=3 2024-03-04T18:04:04.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.298+0000 7fcbf7245980 20 add_watcher() i=4 2024-03-04T18:04:04.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.298+0000 7fcbf7245980 20 add_watcher() i=5 2024-03-04T18:04:04.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.298+0000 7fcbf7245980 20 add_watcher() i=6 2024-03-04T18:04:04.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.302+0000 7fcbf7245980 20 add_watcher() i=7 2024-03-04T18:04:04.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.302+0000 7fcbf7245980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:04.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.306+0000 7fcaaafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.300 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.306+0000 7fcbf7245980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:04.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.306+0000 7fcbf7245980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:04.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.306+0000 7fcbf7245980 5 note: GC not initialized 2024-03-04T18:04:04.301 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.306+0000 7fcaa9fe3640 20 reqs_thread_entry: start 2024-03-04T18:04:04.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.306+0000 7fcbf7245980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:04.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.306+0000 7fcbf7245980 20 Filter name: none 2024-03-04T18:04:04.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.306+0000 7fca1b7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:04.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.306+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.310+0000 7fcbf7245980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:04.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.310+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.310+0000 7fcbf7245980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:04.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.310+0000 7fcbf7245980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:04.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.310+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.310+0000 7fcbf7245980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:04.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.310+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.310+0000 7fcbf7245980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:04.306 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.310+0000 7fcbf7245980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:04.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.314+0000 7fcbf7245980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.309 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.314+0000 7fcbf7245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.311 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.318+0000 7fcaaafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.322+0000 7fcbf7245980 20 remove_watcher() i=0 2024-03-04T18:04:04.316 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.322+0000 7fcbf7245980 2 removed watcher, disabling cache 2024-03-04T18:04:04.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.322+0000 7fcbf7245980 20 remove_watcher() i=1 2024-03-04T18:04:04.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.326+0000 7fcbf7245980 20 remove_watcher() i=2 2024-03-04T18:04:04.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.326+0000 7fcbf7245980 20 remove_watcher() i=3 2024-03-04T18:04:04.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.330+0000 7fcbf7245980 20 remove_watcher() i=4 2024-03-04T18:04:04.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.330+0000 7fcbf7245980 20 remove_watcher() i=5 2024-03-04T18:04:04.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.334+0000 7fcbf7245980 20 remove_watcher() i=6 2024-03-04T18:04:04.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.338+0000 7fcbf7245980 20 remove_watcher() i=7 2024-03-04T18:04:04.349 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:04.349 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:04.349 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-set-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.20 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:04:04.349 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:04.349 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout --inject-error-code 125 2024-03-04T18:04:04.389 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:04.389 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:04.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.438+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 realm 2024-03-04T18:04:04.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:04.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:04.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.442+0000 7f7700466980 20 searching for the correct realm 2024-03-04T18:04:04.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.450+0000 7f76f577d640 0 -- 172.21.15.47:0/2127352287 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f76e8008040 msgr2=0x7f76e8066580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:04.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.666+0000 7f76fe77f640 0 -- 172.21.15.47:0/2127352287 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x562569693e60 msgr2=0x5625696962d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:04.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:04.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:04.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:04.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:04.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:04.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:04.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:04.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 zone default found 2024-03-04T18:04:04.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 4 Realm: () 2024-03-04T18:04:04.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:04.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:04.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:04.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 zonegroup default 2024-03-04T18:04:04.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.678+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.676 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.682+0000 7f7700466980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.676 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.682+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.676 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.682+0000 7f7700466980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.676 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.682+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.677 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.682+0000 7f7700466980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.677 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.682+0000 7f7700466980 20 started sync module instance, tier type = 2024-03-04T18:04:04.677 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.682+0000 7f7700466980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:04.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.706+0000 7f7700466980 20 add_watcher() i=0 2024-03-04T18:04:04.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.706+0000 7f7700466980 20 add_watcher() i=1 2024-03-04T18:04:04.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.706+0000 7f7700466980 20 add_watcher() i=2 2024-03-04T18:04:04.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.706+0000 7f7700466980 20 add_watcher() i=3 2024-03-04T18:04:04.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.706+0000 7f7700466980 20 add_watcher() i=4 2024-03-04T18:04:04.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.706+0000 7f7700466980 20 add_watcher() i=5 2024-03-04T18:04:04.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.706+0000 7f7700466980 20 add_watcher() i=6 2024-03-04T18:04:04.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.710+0000 7f7700466980 20 add_watcher() i=7 2024-03-04T18:04:04.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.710+0000 7f7700466980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:04.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.714+0000 7f75af7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.714+0000 7f7700466980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:04.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.714+0000 7f7700466980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:04.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.714+0000 7f7700466980 5 note: GC not initialized 2024-03-04T18:04:04.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.714+0000 7f75ae7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:04.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.714+0000 7f7700466980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:04.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.714+0000 7f7700466980 20 Filter name: none 2024-03-04T18:04:04.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.714+0000 7f751ffff640 20 reqs_thread_entry: start 2024-03-04T18:04:04.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-03-04T18:04:04.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:04.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-03-04T18:04:04.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-03-04T18:04:04.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 : miss 2024-03-04T18:04:04.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:04.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 info.flags=0x17 2024-03-04T18:04:04.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 to cache LRU end 2024-03-04T18:04:04.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:04.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:04.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.718+0000 7f7700466980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 2024-03-04T18:04:04.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.726+0000 7f7700466980 -1 Injecting error=-125 at location=set_target_layout 2024-03-04T18:04:04.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f7700466980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 info.flags=0x17 2024-03-04T18:04:04.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f7700466980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 to cache LRU end 2024-03-04T18:04:04.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f7700466980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:04.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f7700466980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20, ofs0, ns] 2024-03-04T18:04:04.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f75e17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595109 cookie 94718682334480 notifier 5915 bl.length()=887 2024-03-04T18:04:04.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f75e17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 info.flags=0x17 2024-03-04T18:04:04.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f75e17fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 to cache LRU end 2024-03-04T18:04:04.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f75e17fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:04.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f7700466980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.20]): orig_sources= new_sources= 2024-03-04T18:04:04.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f7700466980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.20]): potential sources added=[] removed=[] 2024-03-04T18:04:04.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f7700466980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.20]): orig_dests= new_dests= 2024-03-04T18:04:04.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.730+0000 7f7700466980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.20]): potential dests added=[] removed=[] 2024-03-04T18:04:04.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.734+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.0 2024-03-04T18:04:04.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.734+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.0 2024-03-04T18:04:04.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.1 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.1 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.2 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.2 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.3 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.3 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.4 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.4 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.5 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.5 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.6 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.6 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.7 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.7 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.8 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.8 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.9 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.9 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.10 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.10 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.11 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1.11 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.0 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.1 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.2 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.3 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.738+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.4 2024-03-04T18:04:04.735 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.742+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.5 2024-03-04T18:04:04.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.742+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.6 2024-03-04T18:04:04.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.742+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.7 2024-03-04T18:04:04.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.742+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.8 2024-03-04T18:04:04.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.742+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.9 2024-03-04T18:04:04.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.746+0000 7f7700466980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.20.10 2024-03-04T18:04:04.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.750+0000 7f7700466980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:04.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.750+0000 7f7700466980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 2024-03-04T18:04:04.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f7700466980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 info.flags=0x17 2024-03-04T18:04:04.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f7700466980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 to cache LRU end 2024-03-04T18:04:04.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f7700466980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:04.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f7700466980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20, ofs0, ns] 2024-03-04T18:04:04.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f75e0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595110 cookie 94718682334480 notifier 5915 bl.length()=855 2024-03-04T18:04:04.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f75e0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 info.flags=0x17 2024-03-04T18:04:04.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f75e0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.20 to cache LRU end 2024-03-04T18:04:04.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f75e0ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:04.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f7700466980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.20]): orig_sources= new_sources= 2024-03-04T18:04:04.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f7700466980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.20]): potential sources added=[] removed=[] 2024-03-04T18:04:04.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f7700466980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.20]): orig_dests= new_dests= 2024-03-04T18:04:04.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.754+0000 7f7700466980 20 handle_bi_update(): bucket=:error-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.20]): potential dests added=[] removed=[] 2024-03-04T18:04:04.755 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.758+0000 7f7700466980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-03-04T18:04:04.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.762+0000 7f75af7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.770+0000 7f7700466980 20 remove_watcher() i=0 2024-03-04T18:04:04.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.770+0000 7f7700466980 2 removed watcher, disabling cache 2024-03-04T18:04:04.767 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.774+0000 7f7700466980 20 remove_watcher() i=1 2024-03-04T18:04:04.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.774+0000 7f7700466980 20 remove_watcher() i=2 2024-03-04T18:04:04.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.782+0000 7f7700466980 20 remove_watcher() i=3 2024-03-04T18:04:04.780 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.786+0000 7f7700466980 20 remove_watcher() i=4 2024-03-04T18:04:04.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.786+0000 7f7700466980 20 remove_watcher() i=5 2024-03-04T18:04:04.786 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.790+0000 7f7700466980 20 remove_watcher() i=6 2024-03-04T18:04:04.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.794+0000 7f7700466980 20 remove_watcher() i=7 2024-03-04T18:04:04.807 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:04:04.838 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:04.838 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:04.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.886+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.885 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.890+0000 7f580389d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.885 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.890+0000 7f580389d980 20 realm 2024-03-04T18:04:04.886 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.890+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.886 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.890+0000 7f580389d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.886 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.890+0000 7f580389d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:04.886 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.890+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.890+0000 7f580389d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.890+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.894+0000 7f580389d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.894+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.894+0000 7f580389d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:04.888 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.894+0000 7f580389d980 20 searching for the correct realm 2024-03-04T18:04:04.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.902+0000 7f5801bb6640 0 -- 172.21.15.47:0/595547915 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f57ec0fc130 msgr2=0x7f57ec0fd800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:04.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.906+0000 7f58013b5640 0 -- 172.21.15.47:0/595547915 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f57fc017070 msgr2=0x7f57fc05aa50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:04.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:04.908 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:04.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:04.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:04.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:04.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:04.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 zone default found 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 4 Realm: () 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:04.910 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:04.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 zonegroup default 2024-03-04T18:04:04.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.911 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.914+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.918+0000 7f580389d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.918+0000 7f580389d980 20 started sync module instance, tier type = 2024-03-04T18:04:04.912 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.918+0000 7f580389d980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.934+0000 7f580389d980 20 add_watcher() i=0 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.934+0000 7f580389d980 20 add_watcher() i=1 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.934+0000 7f580389d980 20 add_watcher() i=2 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.934+0000 7f580389d980 20 add_watcher() i=3 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.934+0000 7f580389d980 20 add_watcher() i=4 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.934+0000 7f580389d980 20 add_watcher() i=5 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.934+0000 7f580389d980 20 add_watcher() i=6 2024-03-04T18:04:04.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.938+0000 7f580389d980 20 add_watcher() i=7 2024-03-04T18:04:04.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.938+0000 7f580389d980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:04.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.942+0000 7f56b77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:04.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:04.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 5 note: GC not initialized 2024-03-04T18:04:04.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f56b67e4640 20 reqs_thread_entry: start 2024-03-04T18:04:04.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:04.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 Filter name: none 2024-03-04T18:04:04.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f5627fff640 20 reqs_thread_entry: start 2024-03-04T18:04:04.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:04.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:04.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:04.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:04.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.946+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.950+0000 7f580389d980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:04.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.950+0000 7f580389d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:04.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.950+0000 7f580389d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.950+0000 7f580389d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.950+0000 7f56b77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.954+0000 7f580389d980 20 remove_watcher() i=0 2024-03-04T18:04:04.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.954+0000 7f580389d980 2 removed watcher, disabling cache 2024-03-04T18:04:04.953 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.958+0000 7f580389d980 20 remove_watcher() i=1 2024-03-04T18:04:04.955 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.962+0000 7f580389d980 20 remove_watcher() i=2 2024-03-04T18:04:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.962+0000 7f580389d980 20 remove_watcher() i=3 2024-03-04T18:04:04.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.966+0000 7f580389d980 20 remove_watcher() i=4 2024-03-04T18:04:04.964 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.970+0000 7f580389d980 20 remove_watcher() i=5 2024-03-04T18:04:04.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.974+0000 7f580389d980 20 remove_watcher() i=6 2024-03-04T18:04:04.973 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:04.978+0000 7f580389d980 20 remove_watcher() i=7 2024-03-04T18:04:04.987 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:04.987 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:04.987 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-set-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.20 num_objects: 20 size_kb: 1 num_shards: 12 2024-03-04T18:04:04.987 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:04.989 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:04.989 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:05.011 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.011 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:05.398 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.398 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:05.799 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with abort at set_target_layout 2024-03-04T18:04:05.799 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:05.802 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.802 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:05.936 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.937 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.019 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.019 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.105 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.105 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.192 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.192 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.277 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.277 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.366 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.366 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.452 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.452 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.535 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.536 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.621 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.621 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.708 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.708 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.799 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.799 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.883 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.883 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:06.968 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.968 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:07.053 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.053 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:07.140 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.140 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:07.227 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.227 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:07.317 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.317 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:07.404 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.404 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:07.488 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.488 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:07.575 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.575 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:07.659 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.659 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:07.743 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-03-04T18:04:07.778 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:07.778 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:07.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.826+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 20 realm 2024-03-04T18:04:07.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:07.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:07.827 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.830+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.827 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.834+0000 7fab9dbae980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:07.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.834+0000 7fab9dbae980 20 searching for the correct realm 2024-03-04T18:04:07.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.838+0000 7fab9bec7640 0 -- 172.21.15.47:0/3182328476 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7fab90115070 msgr2=0x7fab900b9a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:07.841 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.846+0000 7fab9b6c6640 0 -- 172.21.15.47:0/3182328476 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fab94008040 msgr2=0x7fab94025400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:07.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:07.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:07.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:07.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:07.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:07.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:07.849 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 zone default found 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 4 Realm: () 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:07.850 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 zonegroup default 2024-03-04T18:04:07.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.854+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.858+0000 7fab9dbae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.858+0000 7fab9dbae980 20 started sync module instance, tier type = 2024-03-04T18:04:07.852 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.858+0000 7fab9dbae980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:07.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.874+0000 7fab9dbae980 20 add_watcher() i=0 2024-03-04T18:04:07.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.874+0000 7fab9dbae980 20 add_watcher() i=1 2024-03-04T18:04:07.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.874+0000 7fab9dbae980 20 add_watcher() i=2 2024-03-04T18:04:07.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.874+0000 7fab9dbae980 20 add_watcher() i=3 2024-03-04T18:04:07.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.874+0000 7fab9dbae980 20 add_watcher() i=4 2024-03-04T18:04:07.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.874+0000 7fab9dbae980 20 add_watcher() i=5 2024-03-04T18:04:07.870 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.874+0000 7fab9dbae980 20 add_watcher() i=6 2024-03-04T18:04:07.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.878+0000 7fab9dbae980 20 add_watcher() i=7 2024-03-04T18:04:07.872 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.878+0000 7fab9dbae980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:07.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7faa51fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:07.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7fab9dbae980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:07.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7fab9dbae980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:07.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7fab9dbae980 5 note: GC not initialized 2024-03-04T18:04:07.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7faa50fe1640 20 reqs_thread_entry: start 2024-03-04T18:04:07.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7fab9dbae980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:07.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7fab9dbae980 20 Filter name: none 2024-03-04T18:04:07.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7fa9ca7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:07.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7fab9dbae980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:07.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.882+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.886+0000 7fab9dbae980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:07.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.886+0000 7fab9dbae980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:07.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.886+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.886+0000 7fab9dbae980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:07.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.886+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.886+0000 7fab9dbae980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:07.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.886+0000 7fab9dbae980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:07.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.886+0000 7fab9dbae980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.886+0000 7fab9dbae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.890+0000 7faa51fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:07.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.894+0000 7fab9dbae980 20 remove_watcher() i=0 2024-03-04T18:04:07.890 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.894+0000 7fab9dbae980 2 removed watcher, disabling cache 2024-03-04T18:04:07.892 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.898+0000 7fab9dbae980 20 remove_watcher() i=1 2024-03-04T18:04:07.895 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.898+0000 7fab9dbae980 20 remove_watcher() i=2 2024-03-04T18:04:07.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.902+0000 7fab9dbae980 20 remove_watcher() i=3 2024-03-04T18:04:07.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.906+0000 7fab9dbae980 20 remove_watcher() i=4 2024-03-04T18:04:07.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.906+0000 7fab9dbae980 20 remove_watcher() i=5 2024-03-04T18:04:07.904 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.910+0000 7fab9dbae980 20 remove_watcher() i=6 2024-03-04T18:04:07.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:07.914+0000 7fab9dbae980 20 remove_watcher() i=7 2024-03-04T18:04:07.925 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:07.925 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:07.925 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-set-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.21 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:04:07.925 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:07.925 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at set_target_layout 2024-03-04T18:04:07.962 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:07.962 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:08.007 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.014+0000 7f036d7c4980 20 realm 2024-03-04T18:04:08.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.014+0000 7f036d7c4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:08.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.018+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.018+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.018+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:08.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.018+0000 7f036d7c4980 20 searching for the correct realm 2024-03-04T18:04:08.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.022+0000 7f036b2dc640 0 -- 172.21.15.47:0/3822568375 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f035c017070 msgr2=0x7f035c00bdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.030+0000 7f036b2dc640 0 -- 172.21.15.47:0/3822568375 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f03640b8040 msgr2=0x7f03640f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:08.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:08.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:08.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:08.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:08.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:08.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.245 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.245 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.245 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.245 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:08.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 zone default found 2024-03-04T18:04:08.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 4 Realm: () 2024-03-04T18:04:08.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:08.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:08.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:08.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 zonegroup default 2024-03-04T18:04:08.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.254+0000 7f036d7c4980 20 started sync module instance, tier type = 2024-03-04T18:04:08.250 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.254+0000 7f036d7c4980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:08.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=0 2024-03-04T18:04:08.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=1 2024-03-04T18:04:08.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=2 2024-03-04T18:04:08.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=3 2024-03-04T18:04:08.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=4 2024-03-04T18:04:08.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=5 2024-03-04T18:04:08.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=6 2024-03-04T18:04:08.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.282+0000 7f036d7c4980 20 add_watcher() i=7 2024-03-04T18:04:08.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.282+0000 7f036d7c4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:08.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f02227e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f036d7c4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:08.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f036d7c4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:08.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f036d7c4980 5 note: GC not initialized 2024-03-04T18:04:08.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f02217e2640 20 reqs_thread_entry: start 2024-03-04T18:04:08.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f036d7c4980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:08.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f036d7c4980 20 Filter name: none 2024-03-04T18:04:08.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f021a7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:08.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f036d7c4980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:04:08.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.286+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:08.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:04:08.283 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:04:08.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 : miss 2024-03-04T18:04:08.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:08.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 info.flags=0x17 2024-03-04T18:04:08.284 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to cache LRU end 2024-03-04T18:04:08.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:08.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:08.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.290+0000 7f036d7c4980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 2024-03-04T18:04:08.292 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f036d7c4980 time 2024-03-04T18:04:08.300282+0000 2024-03-04T18:04:08.292 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:08.293 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:08.293 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f0371099d7b] 2024-03-04T18:04:08.293 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f0371099f2f] 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5600e61a3281] 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: 4: radosgw-admin(+0x8c3b2a) [0x5600e61aab2a] 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x5600e61afc93] 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: 6: main() 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f036ff04d90] 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: 8: __libc_start_main() 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: 9: _start() 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr:*** Caught signal (Aborted) ** 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7f036d7c4980 thread_name:radosgw-admin 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.298+0000 7f036d7c4980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f036d7c4980 time 2024-03-04T18:04:08.300282+0000 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.294 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f0371099d7b] 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f0371099f2f] 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5600e61a3281] 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 4: radosgw-admin(+0x8c3b2a) [0x5600e61aab2a] 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x5600e61afc93] 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 6: main() 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f036ff04d90] 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 8: __libc_start_main() 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 9: _start() 2024-03-04T18:04:08.295 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.312 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:08.312 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f036ff1d520] 2024-03-04T18:04:08.312 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:04:08.312 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f0371099dd5] 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f0371099f2f] 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5600e61a3281] 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 8: radosgw-admin(+0x8c3b2a) [0x5600e61aab2a] 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x5600e61afc93] 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 10: main() 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f036ff04d90] 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 12: __libc_start_main() 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 13: _start() 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.298+0000 7f036d7c4980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7f036d7c4980 thread_name:radosgw-admin 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f036ff1d520] 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f0371099dd5] 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f0371099f2f] 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5600e61a3281] 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 8: radosgw-admin(+0x8c3b2a) [0x5600e61aab2a] 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x5600e61afc93] 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 10: main() 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f036ff04d90] 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 12: __libc_start_main() 2024-03-04T18:04:08.314 INFO:tasks.workunit.client.0.smithi047.stderr: 13: _start() 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr:--- begin dump of recent events --- 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -499> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -498> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -497> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -496> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): get_auth_request con 0x5600e7ff7170 auth_method 0 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -495> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -494> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -493> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -492> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -491> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.315 INFO:tasks.workunit.client.0.smithi047.stderr: -490> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -489> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -488> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -487> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -486> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): handle_auth_done global_id 5951 payload 274 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -485> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -484> 2024-03-04T18:04:07.990+0000 7f036b2dc640 1 monclient: found mon.noname-c 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -483> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -482> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -481> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr: -480> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: dump: 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.316 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr: -479> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr: -478> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr: -477> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr: -476> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr: -475> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr: -474> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: dump: 2024-03-04T18:04:08.317 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr: -473> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 monclient: authenticate success, global_id 5951 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr: -472> 2024-03-04T18:04:07.990+0000 7f036d7c4980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr: -471> 2024-03-04T18:04:07.990+0000 7f036d7c4980 4 set_mon_vals no callback set 2024-03-04T18:04:08.318 INFO:tasks.workunit.client.0.smithi047.stderr: -470> 2024-03-04T18:04:07.990+0000 7f036d7c4980 10 monclient: shutdown 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr: -469> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 asok(0x5600e7de3130) unregister_commands rotate-key 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr: -468> 2024-03-04T18:04:07.990+0000 7f036d7c4980 10 monclient: build_initial_monmap 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr: -467> 2024-03-04T18:04:07.990+0000 7f036d7c4980 1 build_initial for_mkfs: 0 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr: -466> 2024-03-04T18:04:07.990+0000 7f036d7c4980 10 monclient: monmap: 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:08.319 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -465> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding auth protocol: cephx 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -464> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding auth protocol: cephx 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -463> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding auth protocol: cephx 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -462> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -461> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -460> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -459> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -458> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -457> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -456> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -455> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -454> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.320 INFO:tasks.workunit.client.0.smithi047.stderr: -453> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -452> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -451> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -450> 2024-03-04T18:04:07.994+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -449> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: starting msgr at 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -448> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: starting objecter 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -447> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command objecter_requests hook 0x5600e7fb1960 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -446> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: setting wanted keys 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -445> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: calling monclient init 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -444> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: init 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -443> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding auth protocol: cephx 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -442> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding auth protocol: cephx 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -441> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding auth protocol: cephx 2024-03-04T18:04:08.321 INFO:tasks.workunit.client.0.smithi047.stderr: -440> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -439> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -438> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -437> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -436> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -435> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -434> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -433> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -432> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -431> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -430> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -429> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -428> 2024-03-04T18:04:07.994+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.322 INFO:tasks.workunit.client.0.smithi047.stderr: -427> 2024-03-04T18:04:07.994+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -426> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command rotate-key hook 0x5600e7fc8588 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -425> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -424> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -423> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-b con 0x5600e7fceca0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -422> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-a con 0x5600e7ff7170 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -421> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-c con 0x7f0358003be0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -420> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -419> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -418> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -417> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -416> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): authenticate will time out at 2419.169023s 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -415> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): get_auth_request con 0x5600e7ff7170 auth_method 0 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -414> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.323 INFO:tasks.workunit.client.0.smithi047.stderr: -413> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -412> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -411> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): get_auth_request con 0x7f0358003be0 auth_method 0 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -410> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -409> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -408> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -407> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -406> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -405> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -404> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -403> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -402> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -401> 2024-03-04T18:04:07.994+0000 7f036b2dc640 10 monclient(hunting): get_auth_request con 0x5600e7fceca0 auth_method 0 2024-03-04T18:04:08.324 INFO:tasks.workunit.client.0.smithi047.stderr: -400> 2024-03-04T18:04:07.994+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr: -399> 2024-03-04T18:04:07.994+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr: -398> 2024-03-04T18:04:07.994+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr: -397> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): handle_auth_done global_id 6309 payload 824 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr: -396> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr: -395> 2024-03-04T18:04:07.994+0000 7f0362adb640 1 monclient: found mon.noname-a 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr: -394> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr: -393> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr: -392> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr: -391> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: dump: 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.325 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: -390> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: -389> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: -388> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 monclient: authenticate success, global_id 6309 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: -387> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: -386> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: -385> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: -384> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: -383> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:08.326 INFO:tasks.workunit.client.0.smithi047.stderr: -382> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: dump: 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr: -381> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr: -380> 2024-03-04T18:04:07.994+0000 7f025960a640 4 set_mon_vals no callback set 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr: -379> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.327 INFO:tasks.workunit.client.0.smithi047.stderr: -378> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: init done 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -377> 2024-03-04T18:04:07.998+0000 7f0257e07640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -376> 2024-03-04T18:04:07.998+0000 7f0257e07640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -375> 2024-03-04T18:04:07.998+0000 7f0257e07640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -374> 2024-03-04T18:04:07.998+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f0250065940 auth_method 0 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -373> 2024-03-04T18:04:07.998+0000 7f036badd640 10 monclient: get_auth_request con 0x5600e7fd4250 auth_method 0 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -372> 2024-03-04T18:04:07.998+0000 7f036b2dc640 10 monclient: get_auth_request con 0x5600e8013990 auth_method 0 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -371> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient: get_auth_request con 0x5600e8018a20 auth_method 0 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -370> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: build_initial_monmap 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -369> 2024-03-04T18:04:08.002+0000 7f036d7c4980 1 build_initial for_mkfs: 0 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr: -368> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: monmap: 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:08.328 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr: -367> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding auth protocol: cephx 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr: -366> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding auth protocol: cephx 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr: -365> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding auth protocol: cephx 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr: -364> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr: -363> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.329 INFO:tasks.workunit.client.0.smithi047.stderr: -362> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -361> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -360> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -359> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -358> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -357> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -356> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -355> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -354> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -353> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -352> 2024-03-04T18:04:08.002+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -351> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command objecter_requests cmddesc objecter_requests hook 0x5600e8021d90 EEXIST 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -350> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: init 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -349> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding auth protocol: cephx 2024-03-04T18:04:08.330 INFO:tasks.workunit.client.0.smithi047.stderr: -348> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding auth protocol: cephx 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -347> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding auth protocol: cephx 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -346> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -345> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -344> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -343> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -342> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -341> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -340> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -339> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -338> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -337> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -336> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -335> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.331 INFO:tasks.workunit.client.0.smithi047.stderr: -334> 2024-03-04T18:04:08.002+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -333> 2024-03-04T18:04:08.002+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -332> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command rotate-key cmddesc rotate-key hook 0x5600e801b8b0 EEXIST 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -331> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -330> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -329> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-b con 0x5600e80469f0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -328> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-a con 0x5600e8046dd0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -327> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-c con 0x5600e8041f90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -326> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -325> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -324> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -323> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -322> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): authenticate will time out at 2419.178350s 2024-03-04T18:04:08.332 INFO:tasks.workunit.client.0.smithi047.stderr: -321> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): get_auth_request con 0x5600e8046dd0 auth_method 0 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -320> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -319> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -318> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -317> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): get_auth_request con 0x5600e8041f90 auth_method 0 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -316> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -315> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -314> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -313> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -312> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -311> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -310> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -309> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.333 INFO:tasks.workunit.client.0.smithi047.stderr: -308> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -307> 2024-03-04T18:04:08.002+0000 7f036b2dc640 10 monclient(hunting): get_auth_request con 0x5600e80469f0 auth_method 0 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -306> 2024-03-04T18:04:08.002+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -305> 2024-03-04T18:04:08.002+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -304> 2024-03-04T18:04:08.002+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -303> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): handle_auth_done global_id 5957 payload 824 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -302> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -301> 2024-03-04T18:04:08.002+0000 7f036badd640 1 monclient: found mon.noname-c 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -300> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -299> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -298> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr: -297> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: dump: 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.334 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr: -296> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr: -295> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr: -294> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 monclient: authenticate success, global_id 5957 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr: -293> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr: -292> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:08.335 INFO:tasks.workunit.client.0.smithi047.stderr: -291> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr: -290> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr: -289> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr: -288> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: dump: 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.336 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -287> 2024-03-04T18:04:08.002+0000 7f036c2de640 4 set_mon_vals no callback set 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -286> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -285> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -284> 2024-03-04T18:04:08.006+0000 7f0255602640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -283> 2024-03-04T18:04:08.006+0000 7f0255602640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -282> 2024-03-04T18:04:08.006+0000 7f0255602640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -281> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f0244065880 auth_method 0 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -280> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: get_monmap_and_config 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -279> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: build_initial_monmap 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -278> 2024-03-04T18:04:08.006+0000 7f036d7c4980 1 build_initial for_mkfs: 0 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr: -277> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: monmap: 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:08.337 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr: -276> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr: -275> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr: -274> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr: -273> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr: -272> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr: -271> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.338 INFO:tasks.workunit.client.0.smithi047.stderr: -270> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -269> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -268> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -267> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -266> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -265> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -264> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -263> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -262> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -261> 2024-03-04T18:04:08.006+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -260> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: init 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -259> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding auth protocol: cephx 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -258> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding auth protocol: cephx 2024-03-04T18:04:08.339 INFO:tasks.workunit.client.0.smithi047.stderr: -257> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding auth protocol: cephx 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -256> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -255> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -254> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -253> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -252> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -251> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -250> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -249> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -248> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -247> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -246> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.340 INFO:tasks.workunit.client.0.smithi047.stderr: -245> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -244> 2024-03-04T18:04:08.006+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -243> 2024-03-04T18:04:08.006+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -242> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command rotate-key cmddesc rotate-key hook 0x7fffa91416f8 EEXIST 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -241> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -240> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -239> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-a con 0x5600e8098d10 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -238> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-c con 0x5600e8094220 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -237> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-b con 0x5600e809df00 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -236> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -235> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -234> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -233> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -232> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): authenticate will time out at 2419.182675s 2024-03-04T18:04:08.341 INFO:tasks.workunit.client.0.smithi047.stderr: -231> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): get_auth_request con 0x5600e8098d10 auth_method 0 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -230> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -229> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -228> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -227> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): get_auth_request con 0x5600e8094220 auth_method 0 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -226> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -225> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -224> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -223> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -222> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -221> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -220> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -219> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.342 INFO:tasks.workunit.client.0.smithi047.stderr: -218> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr: -217> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): handle_auth_done global_id 6315 payload 274 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr: -216> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr: -215> 2024-03-04T18:04:08.006+0000 7f036b2dc640 1 monclient: found mon.noname-a 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr: -214> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr: -213> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr: -212> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr: -211> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: dump: 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.343 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: -210> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: -209> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: -208> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: -207> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 monclient: authenticate success, global_id 6315 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: -206> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: -205> 2024-03-04T18:04:08.006+0000 7f036d7c4980 4 set_mon_vals no callback set 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: -204> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: -203> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:08.344 INFO:tasks.workunit.client.0.smithi047.stderr: -202> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: dump: 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr: -201> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: shutdown 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr: -200> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: build_initial_monmap 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr: -199> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 build_initial for_mkfs: 0 2024-03-04T18:04:08.345 INFO:tasks.workunit.client.0.smithi047.stderr: -198> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: monmap: 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr: -197> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr: -196> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.346 INFO:tasks.workunit.client.0.smithi047.stderr: -195> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -194> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -193> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -192> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -191> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -190> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -189> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -188> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -187> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -186> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -185> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -184> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -183> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -182> 2024-03-04T18:04:08.010+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.347 INFO:tasks.workunit.client.0.smithi047.stderr: -181> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: starting msgr at 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -180> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: starting objecter 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -179> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command objecter_requests cmddesc objecter_requests hook 0x5600e805c5a0 EEXIST 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -178> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: setting wanted keys 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -177> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: calling monclient init 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -176> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: init 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -175> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding auth protocol: cephx 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -174> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding auth protocol: cephx 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -173> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding auth protocol: cephx 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -172> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -171> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -170> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -169> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.348 INFO:tasks.workunit.client.0.smithi047.stderr: -168> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -167> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -166> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -165> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -164> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -163> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -162> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -161> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -160> 2024-03-04T18:04:08.010+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -159> 2024-03-04T18:04:08.010+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -158> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command rotate-key cmddesc rotate-key hook 0x5600e8063858 EEXIST 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -157> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -156> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:04:08.349 INFO:tasks.workunit.client.0.smithi047.stderr: -155> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-a con 0x5600e8094220 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -154> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-c con 0x5600e8098d10 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -153> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-b con 0x5600e809df00 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -152> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -151> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -150> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -149> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -148> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): authenticate will time out at 2419.185491s 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -147> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): get_auth_request con 0x5600e8094220 auth_method 0 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -146> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -145> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -144> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -143> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): get_auth_request con 0x5600e8098d10 auth_method 0 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -142> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi047.stderr: -141> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -140> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -139> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -138> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -137> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -136> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -135> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -134> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -133> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): handle_auth_done global_id 6318 payload 824 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -132> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -131> 2024-03-04T18:04:08.010+0000 7f036b2dc640 1 monclient: found mon.noname-a 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -130> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -129> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -128> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:08.351 INFO:tasks.workunit.client.0.smithi047.stderr: -127> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: dump: 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr: -126> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr: -125> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.352 INFO:tasks.workunit.client.0.smithi047.stderr: -124> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr: -123> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr: -122> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr: -121> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: dump: 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.353 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -120> 2024-03-04T18:04:08.010+0000 7f024effd640 4 set_mon_vals no callback set 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -119> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 monclient: authenticate success, global_id 6318 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -118> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -117> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -116> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -115> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -114> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: init done 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -113> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cr dump hook 0x5600e8092e18 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -112> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -111> 2024-03-04T18:04:08.014+0000 7f024d7fa640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -110> 2024-03-04T18:04:08.014+0000 7f024d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -109> 2024-03-04T18:04:08.014+0000 7f024d7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.354 INFO:tasks.workunit.client.0.smithi047.stderr: -108> 2024-03-04T18:04:08.014+0000 7f0362adb640 10 monclient: get_auth_request con 0x7f0234065ae0 auth_method 0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -107> 2024-03-04T18:04:08.014+0000 7f036badd640 10 monclient: get_auth_request con 0x5600e80b2710 auth_method 0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -106> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -105> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 realm 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -104> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -103> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -102> 2024-03-04T18:04:08.014+0000 7f036d7c4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -101> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -100> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -99> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -98> 2024-03-04T18:04:08.014+0000 7f036b2dc640 10 monclient: get_auth_request con 0x5600e80b6940 auth_method 0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -97> 2024-03-04T18:04:08.018+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -96> 2024-03-04T18:04:08.018+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.355 INFO:tasks.workunit.client.0.smithi047.stderr: -95> 2024-03-04T18:04:08.018+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -94> 2024-03-04T18:04:08.018+0000 7f036d7c4980 20 searching for the correct realm 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -93> 2024-03-04T18:04:08.018+0000 7f0362adb640 10 monclient: get_auth_request con 0x5600e80bbb90 auth_method 0 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -92> 2024-03-04T18:04:08.018+0000 7f036badd640 10 monclient: get_auth_request con 0x7f0354019070 auth_method 0 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -91> 2024-03-04T18:04:08.022+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f035c017070 auth_method 0 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -90> 2024-03-04T18:04:08.022+0000 7f036b2dc640 0 -- 172.21.15.47:0/3822568375 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f035c017070 msgr2=0x7f035c00bdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -89> 2024-03-04T18:04:08.022+0000 7f024d7fa640 1 client.6318.objecter ms_handle_reset 0x7f035c017070 session 0x7f035c04e070 osd.0 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -88> 2024-03-04T18:04:08.022+0000 7f024d7fa640 10 monclient: _renew_subs 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -87> 2024-03-04T18:04:08.022+0000 7f024d7fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -86> 2024-03-04T18:04:08.022+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f0234075e60 auth_method 0 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -85> 2024-03-04T18:04:08.026+0000 7f0362adb640 10 monclient: get_auth_request con 0x7f0354010bb0 auth_method 0 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -84> 2024-03-04T18:04:08.030+0000 7f036badd640 10 monclient: get_auth_request con 0x7f035c042be0 auth_method 0 2024-03-04T18:04:08.356 INFO:tasks.workunit.client.0.smithi047.stderr: -83> 2024-03-04T18:04:08.030+0000 7f036b2dc640 0 -- 172.21.15.47:0/3822568375 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f03640b8040 msgr2=0x7f03640f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -82> 2024-03-04T18:04:08.234+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f03640b8040 auth_method 0 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -81> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -80> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -79> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -78> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -77> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -76> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -75> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -74> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -73> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -72> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -71> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -70> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:08.357 INFO:tasks.workunit.client.0.smithi047.stderr: -69> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 zone default found 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -68> 2024-03-04T18:04:08.250+0000 7f036d7c4980 4 Realm: () 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -67> 2024-03-04T18:04:08.250+0000 7f036d7c4980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -66> 2024-03-04T18:04:08.250+0000 7f036d7c4980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -65> 2024-03-04T18:04:08.250+0000 7f036d7c4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -64> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 zonegroup default 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -63> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -62> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -61> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -60> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -59> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -58> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -57> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 started sync module instance, tier type = 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -56> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:08.358 INFO:tasks.workunit.client.0.smithi047.stderr: -55> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=0 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -54> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=1 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -53> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=2 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -52> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=3 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -51> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=4 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -50> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=5 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -49> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=6 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -48> 2024-03-04T18:04:08.282+0000 7f036d7c4980 20 add_watcher() i=7 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -47> 2024-03-04T18:04:08.282+0000 7f036d7c4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -46> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding auth protocol: cephx 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -45> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding auth protocol: cephx 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -44> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding auth protocol: cephx 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -43> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.359 INFO:tasks.workunit.client.0.smithi047.stderr: -42> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -41> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -40> 2024-03-04T18:04:08.286+0000 7f02227e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -39> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -38> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -37> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -36> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -35> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -34> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -33> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -32> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -31> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -30> 2024-03-04T18:04:08.286+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -29> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:08.360 INFO:tasks.workunit.client.0.smithi047.stderr: -28> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -27> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cache list hook 0x5600e809acb0 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -26> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cache inspect hook 0x5600e809acb0 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -25> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cache erase hook 0x5600e809acb0 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -24> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cache zap hook 0x5600e809acb0 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -23> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 note: GC not initialized 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -22> 2024-03-04T18:04:08.286+0000 7f02217e2640 20 reqs_thread_entry: start 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -21> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command sync trace show hook 0x7f036400b640 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -20> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command sync trace history hook 0x7f036400b640 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -19> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command sync trace active hook 0x7f036400b640 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -18> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command sync trace active_short hook 0x7f036400b640 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -17> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -16> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 Filter name: none 2024-03-04T18:04:08.361 INFO:tasks.workunit.client.0.smithi047.stderr: -15> 2024-03-04T18:04:08.286+0000 7f021a7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -14> 2024-03-04T18:04:08.286+0000 7f036d7c4980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -13> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -12> 2024-03-04T18:04:08.290+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -11> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -10> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -9> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 : miss 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -8> 2024-03-04T18:04:08.290+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -7> 2024-03-04T18:04:08.290+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -6> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 info.flags=0x17 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -5> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to cache LRU end 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -4> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -3> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -2> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 2024-03-04T18:04:08.362 INFO:tasks.workunit.client.0.smithi047.stderr: -1> 2024-03-04T18:04:08.298+0000 7f036d7c4980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f036d7c4980 time 2024-03-04T18:04:08.300282+0000 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f0371099d7b] 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f0371099f2f] 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5600e61a3281] 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 4: radosgw-admin(+0x8c3b2a) [0x5600e61aab2a] 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x5600e61afc93] 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 6: main() 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f036ff04d90] 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 8: __libc_start_main() 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 9: _start() 2024-03-04T18:04:08.363 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 0> 2024-03-04T18:04:08.298+0000 7f036d7c4980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7f036d7c4980 thread_name:radosgw-admin 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f036ff1d520] 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f0371099dd5] 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f0371099f2f] 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5600e61a3281] 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 8: radosgw-admin(+0x8c3b2a) [0x5600e61aab2a] 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x5600e61afc93] 2024-03-04T18:04:08.364 INFO:tasks.workunit.client.0.smithi047.stderr: 10: main() 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f036ff04d90] 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 12: __libc_start_main() 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 13: _start() 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr:--- logging levels --- 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 none 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 lockdep 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 context 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 crush 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_balancer 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_locker 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log 2024-03-04T18:04:08.365 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log_expire 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_migrator 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 buffer 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 timer 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 filer 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 striper 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 objecter 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rados 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_mirror 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_replay 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_pwl 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 journaler 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objectcacher 2024-03-04T18:04:08.366 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 client 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 osd 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 optracker 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objclass 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 filestore 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 journal 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 ms 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mon 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 0/10 monc 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 paxos 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 tp 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 auth 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 crypto 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 finisher 2024-03-04T18:04:08.367 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 reserver 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 heartbeatmap 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 perfcounter 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 20/20 rgw 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_sync 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_datacache 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_access 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_flight 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 javaclient 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 asok 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 throttle 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 refs 2024-03-04T18:04:08.368 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 compressor 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluestore 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluefs 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 bdev 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 kstore 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 4/ 5 rocksdb 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 fuse 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 2/ 5 mgr 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mgrc 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 dpdk 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 eventtrace 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 prioritycache 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 test 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:04:08.369 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephsqlite 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_onode 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_odata 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_omap 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_tm 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_t 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_epm 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_lba 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cache 2024-03-04T18:04:08.370 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_journal 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_device 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_backref 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 alienstore 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mclock 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cyanstore 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 ceph_exporter 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 memstore 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 trace 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: -2/-2 (syslog threshold) 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 99/99 (stderr threshold) 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 7f021a7fc640 / http_manager 2024-03-04T18:04:08.371 INFO:tasks.workunit.client.0.smithi047.stderr: 7f02217e2640 / http_manager 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f02227e4640 / rgw_dt_lg_renew 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f024d7fa640 / ms_dispatch 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f024effd640 / io_context_pool 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f0255602640 / ms_dispatch 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f0257e07640 / ms_dispatch 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f025960a640 / io_context_pool 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f0362adb640 / msgr-worker-2 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f036b2dc640 / msgr-worker-1 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f036badd640 / msgr-worker-0 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f036c2de640 / io_context_pool 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: 7f036d7c4980 / radosgw-admin 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: max_recent 500 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: max_new 1000 2024-03-04T18:04:08.372 INFO:tasks.workunit.client.0.smithi047.stderr: log_file /var/log/ceph/ceph-client.admin.79319.log 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr:--- end dump of recent events --- 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr:--- begin dump of recent events --- 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -499> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -498> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -497> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -496> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): get_auth_request con 0x5600e7ff7170 auth_method 0 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -495> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -494> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -493> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -492> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -491> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -490> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.373 INFO:tasks.workunit.client.0.smithi047.stderr: -489> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr: -488> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr: -487> 2024-03-04T18:04:07.990+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr: -486> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient(hunting): handle_auth_done global_id 5951 payload 274 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr: -485> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr: -484> 2024-03-04T18:04:07.990+0000 7f036b2dc640 1 monclient: found mon.noname-c 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr: -483> 2024-03-04T18:04:07.990+0000 7f036b2dc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr: -482> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr: -481> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr: -480> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: dump: 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr: -479> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr: -478> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr: -477> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr: -476> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr: -475> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr: -474> 2024-03-04T18:04:07.990+0000 7f025960a640 10 monclient: dump: 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.375 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr: -473> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 monclient: authenticate success, global_id 5951 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr: -472> 2024-03-04T18:04:07.990+0000 7f036d7c4980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr: -471> 2024-03-04T18:04:07.990+0000 7f036d7c4980 4 set_mon_vals no callback set 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr: -470> 2024-03-04T18:04:07.990+0000 7f036d7c4980 10 monclient: shutdown 2024-03-04T18:04:08.376 INFO:tasks.workunit.client.0.smithi047.stderr: -469> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 asok(0x5600e7de3130) unregister_commands rotate-key 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr: -468> 2024-03-04T18:04:07.990+0000 7f036d7c4980 10 monclient: build_initial_monmap 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr: -467> 2024-03-04T18:04:07.990+0000 7f036d7c4980 1 build_initial for_mkfs: 0 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr: -466> 2024-03-04T18:04:07.990+0000 7f036d7c4980 10 monclient: monmap: 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:08.377 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -465> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding auth protocol: cephx 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -464> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding auth protocol: cephx 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -463> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding auth protocol: cephx 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -462> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -461> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -460> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -459> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -458> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -457> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -456> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -455> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -454> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -453> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.378 INFO:tasks.workunit.client.0.smithi047.stderr: -452> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: crc 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -451> 2024-03-04T18:04:07.990+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fca2a0) adding con mode: secure 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -450> 2024-03-04T18:04:07.994+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -449> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: starting msgr at 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -448> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: starting objecter 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -447> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command objecter_requests hook 0x5600e7fb1960 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -446> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: setting wanted keys 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -445> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: calling monclient init 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -444> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: init 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -443> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding auth protocol: cephx 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -442> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding auth protocol: cephx 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -441> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding auth protocol: cephx 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -440> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -439> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.379 INFO:tasks.workunit.client.0.smithi047.stderr: -438> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -437> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -436> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -435> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -434> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -433> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -432> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -431> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -430> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: crc 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -429> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 AuthRegistry(0x5600e7fc8440) adding con mode: secure 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -428> 2024-03-04T18:04:07.994+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -427> 2024-03-04T18:04:07.994+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -426> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command rotate-key hook 0x5600e7fc8588 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -425> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:08.380 INFO:tasks.workunit.client.0.smithi047.stderr: -424> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -423> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-b con 0x5600e7fceca0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -422> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-a con 0x5600e7ff7170 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -421> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-c con 0x7f0358003be0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -420> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -419> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -418> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -417> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -416> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient(hunting): authenticate will time out at 2419.169023s 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -415> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): get_auth_request con 0x5600e7ff7170 auth_method 0 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -414> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -413> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -412> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.381 INFO:tasks.workunit.client.0.smithi047.stderr: -411> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): get_auth_request con 0x7f0358003be0 auth_method 0 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -410> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -409> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -408> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -407> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -406> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -405> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -404> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -403> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -402> 2024-03-04T18:04:07.994+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -401> 2024-03-04T18:04:07.994+0000 7f036b2dc640 10 monclient(hunting): get_auth_request con 0x5600e7fceca0 auth_method 0 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -400> 2024-03-04T18:04:07.994+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -399> 2024-03-04T18:04:07.994+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -398> 2024-03-04T18:04:07.994+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -397> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient(hunting): handle_auth_done global_id 6309 payload 824 2024-03-04T18:04:08.382 INFO:tasks.workunit.client.0.smithi047.stderr: -396> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr: -395> 2024-03-04T18:04:07.994+0000 7f0362adb640 1 monclient: found mon.noname-a 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr: -394> 2024-03-04T18:04:07.994+0000 7f0362adb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr: -393> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr: -392> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr: -391> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: dump: 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: -390> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: -389> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: -388> 2024-03-04T18:04:07.994+0000 7f036d7c4980 5 monclient: authenticate success, global_id 6309 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: -387> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: -386> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: -385> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: -384> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: -383> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr: -382> 2024-03-04T18:04:07.994+0000 7f0257e07640 10 monclient: dump: 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.384 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr: -381> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr: -380> 2024-03-04T18:04:07.994+0000 7f025960a640 4 set_mon_vals no callback set 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr: -379> 2024-03-04T18:04:07.994+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr: -378> 2024-03-04T18:04:07.994+0000 7f036d7c4980 1 librados: init done 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr: -377> 2024-03-04T18:04:07.998+0000 7f0257e07640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr: -376> 2024-03-04T18:04:07.998+0000 7f0257e07640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.385 INFO:tasks.workunit.client.0.smithi047.stderr: -375> 2024-03-04T18:04:07.998+0000 7f0257e07640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr: -374> 2024-03-04T18:04:07.998+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f0250065940 auth_method 0 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr: -373> 2024-03-04T18:04:07.998+0000 7f036badd640 10 monclient: get_auth_request con 0x5600e7fd4250 auth_method 0 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr: -372> 2024-03-04T18:04:07.998+0000 7f036b2dc640 10 monclient: get_auth_request con 0x5600e8013990 auth_method 0 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr: -371> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient: get_auth_request con 0x5600e8018a20 auth_method 0 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr: -370> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: build_initial_monmap 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr: -369> 2024-03-04T18:04:08.002+0000 7f036d7c4980 1 build_initial for_mkfs: 0 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr: -368> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: monmap: 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.386 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -367> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding auth protocol: cephx 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -366> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding auth protocol: cephx 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -365> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding auth protocol: cephx 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -364> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -363> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -362> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -361> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -360> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -359> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.387 INFO:tasks.workunit.client.0.smithi047.stderr: -358> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -357> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -356> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -355> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -354> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: crc 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -353> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801c5d0) adding con mode: secure 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -352> 2024-03-04T18:04:08.002+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -351> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command objecter_requests cmddesc objecter_requests hook 0x5600e8021d90 EEXIST 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -350> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: init 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -349> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding auth protocol: cephx 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -348> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding auth protocol: cephx 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -347> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding auth protocol: cephx 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -346> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.388 INFO:tasks.workunit.client.0.smithi047.stderr: -345> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -344> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -343> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -342> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -341> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -340> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -339> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -338> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -337> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -336> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: crc 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -335> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 AuthRegistry(0x5600e801b768) adding con mode: secure 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -334> 2024-03-04T18:04:08.002+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -333> 2024-03-04T18:04:08.002+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -332> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command rotate-key cmddesc rotate-key hook 0x5600e801b8b0 EEXIST 2024-03-04T18:04:08.389 INFO:tasks.workunit.client.0.smithi047.stderr: -331> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -330> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -329> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-b con 0x5600e80469f0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -328> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-a con 0x5600e8046dd0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -327> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-c con 0x5600e8041f90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -326> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -325> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -324> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -323> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -322> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient(hunting): authenticate will time out at 2419.178350s 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -321> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): get_auth_request con 0x5600e8046dd0 auth_method 0 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -320> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -319> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -318> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.390 INFO:tasks.workunit.client.0.smithi047.stderr: -317> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): get_auth_request con 0x5600e8041f90 auth_method 0 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -316> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -315> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -314> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -313> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -312> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -311> 2024-03-04T18:04:08.002+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -310> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -309> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -308> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -307> 2024-03-04T18:04:08.002+0000 7f036b2dc640 10 monclient(hunting): get_auth_request con 0x5600e80469f0 auth_method 0 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -306> 2024-03-04T18:04:08.002+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -305> 2024-03-04T18:04:08.002+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -304> 2024-03-04T18:04:08.002+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.391 INFO:tasks.workunit.client.0.smithi047.stderr: -303> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient(hunting): handle_auth_done global_id 5957 payload 824 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr: -302> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr: -301> 2024-03-04T18:04:08.002+0000 7f036badd640 1 monclient: found mon.noname-c 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr: -300> 2024-03-04T18:04:08.002+0000 7f036badd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr: -299> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr: -298> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr: -297> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: dump: 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: -296> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: -295> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: -294> 2024-03-04T18:04:08.002+0000 7f036d7c4980 5 monclient: authenticate success, global_id 5957 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: -293> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: -292> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: -291> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: -290> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: -289> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr: -288> 2024-03-04T18:04:08.002+0000 7f0255602640 10 monclient: dump: 2024-03-04T18:04:08.393 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr: -287> 2024-03-04T18:04:08.002+0000 7f036c2de640 4 set_mon_vals no callback set 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr: -286> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr: -285> 2024-03-04T18:04:08.002+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr: -284> 2024-03-04T18:04:08.006+0000 7f0255602640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:08.394 INFO:tasks.workunit.client.0.smithi047.stderr: -283> 2024-03-04T18:04:08.006+0000 7f0255602640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr: -282> 2024-03-04T18:04:08.006+0000 7f0255602640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr: -281> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f0244065880 auth_method 0 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr: -280> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: get_monmap_and_config 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr: -279> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: build_initial_monmap 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr: -278> 2024-03-04T18:04:08.006+0000 7f036d7c4980 1 build_initial for_mkfs: 0 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr: -277> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: monmap: 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -276> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -275> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -274> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -273> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -272> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -271> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -270> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -269> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -268> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi047.stderr: -267> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -266> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -265> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -264> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -263> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -262> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -261> 2024-03-04T18:04:08.006+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -260> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: init 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -259> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding auth protocol: cephx 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -258> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding auth protocol: cephx 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -257> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding auth protocol: cephx 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -256> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -255> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi047.stderr: -254> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -253> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -252> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -251> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -250> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -249> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -248> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -247> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -246> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: crc 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -245> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 AuthRegistry(0x7fffa91415b0) adding con mode: secure 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -244> 2024-03-04T18:04:08.006+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -243> 2024-03-04T18:04:08.006+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -242> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command rotate-key cmddesc rotate-key hook 0x7fffa91416f8 EEXIST 2024-03-04T18:04:08.398 INFO:tasks.workunit.client.0.smithi047.stderr: -241> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -240> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -239> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-a con 0x5600e8098d10 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -238> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-c con 0x5600e8094220 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -237> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-b con 0x5600e809df00 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -236> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -235> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -234> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -233> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -232> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient(hunting): authenticate will time out at 2419.182675s 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -231> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): get_auth_request con 0x5600e8098d10 auth_method 0 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -230> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -229> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -228> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.399 INFO:tasks.workunit.client.0.smithi047.stderr: -227> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): get_auth_request con 0x5600e8094220 auth_method 0 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -226> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -225> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -224> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -223> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -222> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -221> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -220> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -219> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -218> 2024-03-04T18:04:08.006+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -217> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient(hunting): handle_auth_done global_id 6315 payload 274 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -216> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.400 INFO:tasks.workunit.client.0.smithi047.stderr: -215> 2024-03-04T18:04:08.006+0000 7f036b2dc640 1 monclient: found mon.noname-a 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr: -214> 2024-03-04T18:04:08.006+0000 7f036b2dc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr: -213> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr: -212> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr: -211> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: dump: 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: -210> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: -209> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: -208> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: -207> 2024-03-04T18:04:08.006+0000 7f036d7c4980 5 monclient: authenticate success, global_id 6315 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: -206> 2024-03-04T18:04:08.006+0000 7f036d7c4980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: -205> 2024-03-04T18:04:08.006+0000 7f036d7c4980 4 set_mon_vals no callback set 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: -204> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: -203> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr: -202> 2024-03-04T18:04:08.006+0000 7f024effd640 10 monclient: dump: 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.402 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr: -201> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: shutdown 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr: -200> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: build_initial_monmap 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr: -199> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 build_initial for_mkfs: 0 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr: -198> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: monmap: 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:08.403 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: -197> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: -196> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: -195> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding auth protocol: cephx 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: -194> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: -193> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: -192> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: -191> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: -190> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.404 INFO:tasks.workunit.client.0.smithi047.stderr: -189> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -188> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -187> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -186> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -185> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -184> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: crc 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -183> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8064540) adding con mode: secure 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -182> 2024-03-04T18:04:08.010+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -181> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: starting msgr at 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -180> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: starting objecter 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -179> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command objecter_requests cmddesc objecter_requests hook 0x5600e805c5a0 EEXIST 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -178> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: setting wanted keys 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -177> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: calling monclient init 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -176> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: init 2024-03-04T18:04:08.405 INFO:tasks.workunit.client.0.smithi047.stderr: -175> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding auth protocol: cephx 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -174> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding auth protocol: cephx 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -173> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding auth protocol: cephx 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -172> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -171> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -170> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -169> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -168> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -167> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -166> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -165> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -164> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -163> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -162> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: crc 2024-03-04T18:04:08.406 INFO:tasks.workunit.client.0.smithi047.stderr: -161> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 AuthRegistry(0x5600e8063710) adding con mode: secure 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -160> 2024-03-04T18:04:08.010+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -159> 2024-03-04T18:04:08.010+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -158> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command rotate-key cmddesc rotate-key hook 0x5600e8063858 EEXIST 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -157> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -156> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -155> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-a con 0x5600e8094220 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -154> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-c con 0x5600e8098d10 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -153> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): picked mon.noname-b con 0x5600e809df00 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -152> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -151> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -150> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -149> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:08.407 INFO:tasks.workunit.client.0.smithi047.stderr: -148> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient(hunting): authenticate will time out at 2419.185491s 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -147> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): get_auth_request con 0x5600e8094220 auth_method 0 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -146> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -145> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -144> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -143> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): get_auth_request con 0x5600e8098d10 auth_method 0 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -142> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -141> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -140> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -139> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -138> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -137> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -136> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -135> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:08.408 INFO:tasks.workunit.client.0.smithi047.stderr: -134> 2024-03-04T18:04:08.010+0000 7f0362adb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr: -133> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient(hunting): handle_auth_done global_id 6318 payload 824 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr: -132> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient: _finish_hunting 0 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr: -131> 2024-03-04T18:04:08.010+0000 7f036b2dc640 1 monclient: found mon.noname-a 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr: -130> 2024-03-04T18:04:08.010+0000 7f036b2dc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr: -129> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr: -128> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr: -127> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: dump: 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.409 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr: -126> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: _finish_auth 0 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr: -125> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: _check_auth_tickets 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr: -124> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr: -123> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr: -122> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr: -121> 2024-03-04T18:04:08.010+0000 7f024d7fa640 10 monclient: dump: 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.410 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr: -120> 2024-03-04T18:04:08.010+0000 7f024effd640 4 set_mon_vals no callback set 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr: -119> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 monclient: authenticate success, global_id 6318 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr: -118> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr: -117> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr: -116> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _renew_subs 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr: -115> 2024-03-04T18:04:08.010+0000 7f036d7c4980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr: -114> 2024-03-04T18:04:08.010+0000 7f036d7c4980 1 librados: init done 2024-03-04T18:04:08.411 INFO:tasks.workunit.client.0.smithi047.stderr: -113> 2024-03-04T18:04:08.010+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cr dump hook 0x5600e8092e18 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -112> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -111> 2024-03-04T18:04:08.014+0000 7f024d7fa640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -110> 2024-03-04T18:04:08.014+0000 7f024d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -109> 2024-03-04T18:04:08.014+0000 7f024d7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -108> 2024-03-04T18:04:08.014+0000 7f0362adb640 10 monclient: get_auth_request con 0x7f0234065ae0 auth_method 0 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -107> 2024-03-04T18:04:08.014+0000 7f036badd640 10 monclient: get_auth_request con 0x5600e80b2710 auth_method 0 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -106> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -105> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 realm 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -104> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -103> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -102> 2024-03-04T18:04:08.014+0000 7f036d7c4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -101> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.412 INFO:tasks.workunit.client.0.smithi047.stderr: -100> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -99> 2024-03-04T18:04:08.014+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -98> 2024-03-04T18:04:08.014+0000 7f036b2dc640 10 monclient: get_auth_request con 0x5600e80b6940 auth_method 0 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -97> 2024-03-04T18:04:08.018+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -96> 2024-03-04T18:04:08.018+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -95> 2024-03-04T18:04:08.018+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -94> 2024-03-04T18:04:08.018+0000 7f036d7c4980 20 searching for the correct realm 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -93> 2024-03-04T18:04:08.018+0000 7f0362adb640 10 monclient: get_auth_request con 0x5600e80bbb90 auth_method 0 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -92> 2024-03-04T18:04:08.018+0000 7f036badd640 10 monclient: get_auth_request con 0x7f0354019070 auth_method 0 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -91> 2024-03-04T18:04:08.022+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f035c017070 auth_method 0 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -90> 2024-03-04T18:04:08.022+0000 7f036b2dc640 0 -- 172.21.15.47:0/3822568375 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f035c017070 msgr2=0x7f035c00bdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -89> 2024-03-04T18:04:08.022+0000 7f024d7fa640 1 client.6318.objecter ms_handle_reset 0x7f035c017070 session 0x7f035c04e070 osd.0 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -88> 2024-03-04T18:04:08.022+0000 7f024d7fa640 10 monclient: _renew_subs 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -87> 2024-03-04T18:04:08.022+0000 7f024d7fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:08.413 INFO:tasks.workunit.client.0.smithi047.stderr: -86> 2024-03-04T18:04:08.022+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f0234075e60 auth_method 0 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -85> 2024-03-04T18:04:08.026+0000 7f0362adb640 10 monclient: get_auth_request con 0x7f0354010bb0 auth_method 0 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -84> 2024-03-04T18:04:08.030+0000 7f036badd640 10 monclient: get_auth_request con 0x7f035c042be0 auth_method 0 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -83> 2024-03-04T18:04:08.030+0000 7f036b2dc640 0 -- 172.21.15.47:0/3822568375 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f03640b8040 msgr2=0x7f03640f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -82> 2024-03-04T18:04:08.234+0000 7f036b2dc640 10 monclient: get_auth_request con 0x7f03640b8040 auth_method 0 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -81> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -80> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -79> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -78> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -77> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -76> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -75> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -74> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.414 INFO:tasks.workunit.client.0.smithi047.stderr: -73> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -72> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -71> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -70> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -69> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 zone default found 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -68> 2024-03-04T18:04:08.250+0000 7f036d7c4980 4 Realm: () 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -67> 2024-03-04T18:04:08.250+0000 7f036d7c4980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -66> 2024-03-04T18:04:08.250+0000 7f036d7c4980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -65> 2024-03-04T18:04:08.250+0000 7f036d7c4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -64> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 zonegroup default 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -63> 2024-03-04T18:04:08.250+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -62> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -61> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -60> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.415 INFO:tasks.workunit.client.0.smithi047.stderr: -59> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -58> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -57> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 started sync module instance, tier type = 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -56> 2024-03-04T18:04:08.254+0000 7f036d7c4980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -55> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=0 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -54> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=1 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -53> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=2 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -52> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=3 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -51> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=4 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -50> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=5 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -49> 2024-03-04T18:04:08.278+0000 7f036d7c4980 20 add_watcher() i=6 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -48> 2024-03-04T18:04:08.282+0000 7f036d7c4980 20 add_watcher() i=7 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -47> 2024-03-04T18:04:08.282+0000 7f036d7c4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -46> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding auth protocol: cephx 2024-03-04T18:04:08.416 INFO:tasks.workunit.client.0.smithi047.stderr: -45> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding auth protocol: cephx 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -44> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding auth protocol: cephx 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -43> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -42> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -41> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -40> 2024-03-04T18:04:08.286+0000 7f02227e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -39> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -38> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -37> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -36> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -35> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -34> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -33> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.417 INFO:tasks.workunit.client.0.smithi047.stderr: -32> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: crc 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -31> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 AuthRegistry(0x7fffa9141a40) adding con mode: secure 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -30> 2024-03-04T18:04:08.286+0000 7f036d7c4980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -29> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -28> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -27> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cache list hook 0x5600e809acb0 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -26> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cache inspect hook 0x5600e809acb0 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -25> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cache erase hook 0x5600e809acb0 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -24> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command cache zap hook 0x5600e809acb0 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -23> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 note: GC not initialized 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -22> 2024-03-04T18:04:08.286+0000 7f02217e2640 20 reqs_thread_entry: start 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -21> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command sync trace show hook 0x7f036400b640 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -20> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command sync trace history hook 0x7f036400b640 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -19> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command sync trace active hook 0x7f036400b640 2024-03-04T18:04:08.418 INFO:tasks.workunit.client.0.smithi047.stderr: -18> 2024-03-04T18:04:08.286+0000 7f036d7c4980 5 asok(0x5600e7de3130) register_command sync trace active_short hook 0x7f036400b640 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -17> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -16> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 Filter name: none 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -15> 2024-03-04T18:04:08.286+0000 7f021a7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -14> 2024-03-04T18:04:08.286+0000 7f036d7c4980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -13> 2024-03-04T18:04:08.286+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -12> 2024-03-04T18:04:08.290+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -11> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -10> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -9> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 : miss 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -8> 2024-03-04T18:04:08.290+0000 7f036d7c4980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -7> 2024-03-04T18:04:08.290+0000 7f036d7c4980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -6> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 info.flags=0x17 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -5> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to cache LRU end 2024-03-04T18:04:08.419 INFO:tasks.workunit.client.0.smithi047.stderr: -4> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: -3> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: -2> 2024-03-04T18:04:08.290+0000 7f036d7c4980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: -1> 2024-03-04T18:04:08.298+0000 7f036d7c4980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f036d7c4980 time 2024-03-04T18:04:08.300282+0000 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f0371099d7b] 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f0371099f2f] 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5600e61a3281] 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: 4: radosgw-admin(+0x8c3b2a) [0x5600e61aab2a] 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x5600e61afc93] 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: 6: main() 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f036ff04d90] 2024-03-04T18:04:08.420 INFO:tasks.workunit.client.0.smithi047.stderr: 8: __libc_start_main() 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 9: _start() 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 0> 2024-03-04T18:04:08.298+0000 7f036d7c4980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7f036d7c4980 thread_name:radosgw-admin 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f036ff1d520] 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f0371099dd5] 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f0371099f2f] 2024-03-04T18:04:08.421 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5600e61a3281] 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 8: radosgw-admin(+0x8c3b2a) [0x5600e61aab2a] 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x5600e61afc93] 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 10: main() 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f036ff04d90] 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 12: __libc_start_main() 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 13: _start() 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr:--- logging levels --- 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 none 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 lockdep 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 context 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 crush 2024-03-04T18:04:08.422 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_balancer 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_locker 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log_expire 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_migrator 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 buffer 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 timer 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 filer 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 striper 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 objecter 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rados 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_mirror 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_replay 2024-03-04T18:04:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_pwl 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 journaler 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objectcacher 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 client 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 osd 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 optracker 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objclass 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 filestore 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 journal 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 ms 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mon 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 0/10 monc 2024-03-04T18:04:08.424 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 paxos 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 tp 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 auth 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 crypto 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 finisher 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 reserver 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 heartbeatmap 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 perfcounter 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 20/20 rgw 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_sync 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_datacache 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_access 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_flight 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:04:08.425 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 javaclient 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 asok 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 throttle 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 refs 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 compressor 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluestore 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluefs 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 bdev 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 kstore 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 4/ 5 rocksdb 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 fuse 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 2/ 5 mgr 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mgrc 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 dpdk 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 eventtrace 2024-03-04T18:04:08.426 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 prioritycache 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 test 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephsqlite 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_onode 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_odata 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_omap 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_tm 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_t 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_epm 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_lba 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:04:08.427 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cache 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_journal 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_device 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_backref 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 alienstore 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mclock 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cyanstore 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 ceph_exporter 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 memstore 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 trace 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: -2/-2 (syslog threshold) 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr: 99/99 (stderr threshold) 2024-03-04T18:04:08.428 INFO:tasks.workunit.client.0.smithi047.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f021a7fc640 / http_manager 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f02217e2640 / http_manager 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f02227e4640 / rgw_dt_lg_renew 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f024d7fa640 / ms_dispatch 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f024effd640 / io_context_pool 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f0255602640 / ms_dispatch 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f0257e07640 / ms_dispatch 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f025960a640 / io_context_pool 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f0362adb640 / msgr-worker-2 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f036b2dc640 / msgr-worker-1 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f036badd640 / msgr-worker-0 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f036c2de640 / io_context_pool 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: 7f036d7c4980 / radosgw-admin 2024-03-04T18:04:08.429 INFO:tasks.workunit.client.0.smithi047.stderr: max_recent 500 2024-03-04T18:04:08.430 INFO:tasks.workunit.client.0.smithi047.stderr: max_new 1000 2024-03-04T18:04:08.430 INFO:tasks.workunit.client.0.smithi047.stderr: log_file /var/lib/ceph/crash/2024-03-04T18:04:08.303551Z_32ffbb37-b2da-45d9-b78c-3b8c08107444/log 2024-03-04T18:04:08.430 INFO:tasks.workunit.client.0.smithi047.stderr:--- end dump of recent events --- 2024-03-04T18:04:08.430 INFO:tasks.workunit.client.0.smithi047.stderr:Aborted 2024-03-04T18:04:08.430 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.386+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.386+0000 7f56c27da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.386+0000 7f56c27da980 20 realm 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.386+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.386+0000 7f56c27da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.386+0000 7f56c27da980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.386+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.390+0000 7f56c27da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.390+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.390+0000 7f56c27da980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.390+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.390+0000 7f56c27da980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.390+0000 7f56c27da980 20 searching for the correct realm 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.398+0000 7f56c0af3640 0 -- 172.21.15.47:0/2139931954 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f56b4008050 msgr2=0x7f56b403a400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.402+0000 7f56c02f2640 0 -- 172.21.15.47:0/2139931954 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f56b807c0b0 msgr2=0x7f56b80781b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.410+0000 7f56c27da980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.410+0000 7f56c27da980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.410+0000 7f56c27da980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.410+0000 7f56c27da980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.410+0000 7f56c27da980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.410+0000 7f56c27da980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.410+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 zone default found 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 4 Realm: () 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 zonegroup default 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 started sync module instance, tier type = 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.414+0000 7f56c27da980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.434+0000 7f56c27da980 20 add_watcher() i=0 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.434+0000 7f56c27da980 20 add_watcher() i=1 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.434+0000 7f56c27da980 20 add_watcher() i=2 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.434+0000 7f56c27da980 20 add_watcher() i=3 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.434+0000 7f56c27da980 20 add_watcher() i=4 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.434+0000 7f56c27da980 20 add_watcher() i=5 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.434+0000 7f56c27da980 20 add_watcher() i=6 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.434+0000 7f56c27da980 20 add_watcher() i=7 2024-03-04T18:04:08.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.434+0000 7f56c27da980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:08.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.438+0000 7f5576fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.438+0000 7f56c27da980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.438+0000 7f56c27da980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.438+0000 7f56c27da980 5 note: GC not initialized 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.438+0000 7f5575fdb640 20 reqs_thread_entry: start 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.438+0000 7f56c27da980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.438+0000 7f56c27da980 20 Filter name: none 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.438+0000 7f54e77fe640 20 reqs_thread_entry: start 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.438+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.439 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:08.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:08.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:08.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.442+0000 7f56c27da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.440 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.446+0000 7f5576fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.450+0000 7f56c27da980 20 remove_watcher() i=0 2024-03-04T18:04:08.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.450+0000 7f56c27da980 2 removed watcher, disabling cache 2024-03-04T18:04:08.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.450+0000 7f56c27da980 20 remove_watcher() i=1 2024-03-04T18:04:08.447 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.454+0000 7f56c27da980 20 remove_watcher() i=2 2024-03-04T18:04:08.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.454+0000 7f56c27da980 20 remove_watcher() i=3 2024-03-04T18:04:08.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.458+0000 7f56c27da980 20 remove_watcher() i=4 2024-03-04T18:04:08.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.458+0000 7f56c27da980 20 remove_watcher() i=5 2024-03-04T18:04:08.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.462+0000 7f56c27da980 20 remove_watcher() i=6 2024-03-04T18:04:08.459 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.462+0000 7f56c27da980 20 remove_watcher() i=7 2024-03-04T18:04:08.473 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.473 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:08.473 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-set-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.21 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:04:08.473 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:08.474 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:08.475 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:08.508 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:08.508 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:08.565 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:08.601 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:08.602 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:08.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.650+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 realm 2024-03-04T18:04:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:08.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.654+0000 7f0a6ee90980 20 searching for the correct realm 2024-03-04T18:04:08.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.662+0000 7f0a6c9a8640 0 -- 172.21.15.47:0/307758662 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f0a5c0e9440 msgr2=0x7f0a5c0e9880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.670+0000 7f0a6c9a8640 0 -- 172.21.15.47:0/307758662 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f0a64092040 msgr2=0x7f0a640732a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.674+0000 7f0a6ee90980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.674+0000 7f0a6ee90980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.674+0000 7f0a6ee90980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.674+0000 7f0a6ee90980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.674+0000 7f0a6ee90980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.674+0000 7f0a6ee90980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.674+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 zone default found 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 4 Realm: () 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 zonegroup default 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 started sync module instance, tier type = 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.678+0000 7f0a6ee90980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:08.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.698+0000 7f0a6ee90980 20 add_watcher() i=0 2024-03-04T18:04:08.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.698+0000 7f0a6ee90980 20 add_watcher() i=1 2024-03-04T18:04:08.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.698+0000 7f0a6ee90980 20 add_watcher() i=2 2024-03-04T18:04:08.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.698+0000 7f0a6ee90980 20 add_watcher() i=3 2024-03-04T18:04:08.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.698+0000 7f0a6ee90980 20 add_watcher() i=4 2024-03-04T18:04:08.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.698+0000 7f0a6ee90980 20 add_watcher() i=5 2024-03-04T18:04:08.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.698+0000 7f0a6ee90980 20 add_watcher() i=6 2024-03-04T18:04:08.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.698+0000 7f0a6ee90980 20 add_watcher() i=7 2024-03-04T18:04:08.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.698+0000 7f0a6ee90980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:08.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.702+0000 7f0922fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.702+0000 7f0a6ee90980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.702+0000 7f0a6ee90980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.702+0000 7f0a6ee90980 5 note: GC not initialized 2024-03-04T18:04:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0921fe3640 20 reqs_thread_entry: start 2024-03-04T18:04:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 20 Filter name: none 2024-03-04T18:04:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f08937fe640 20 reqs_thread_entry: start 2024-03-04T18:04:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:04:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:04:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:04:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 : miss 2024-03-04T18:04:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:08.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 info.flags=0x17 2024-03-04T18:04:08.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to cache LRU end 2024-03-04T18:04:08.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:08.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:08.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 2024-03-04T18:04:08.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0a6ee90980 0 INFO: RGWReshardLock::lock found lock on abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to be held by another RGW process; skipping for now 2024-03-04T18:04:08.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.706+0000 7f0922fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.714+0000 7f0a6ee90980 20 remove_watcher() i=0 2024-03-04T18:04:08.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.714+0000 7f0a6ee90980 2 removed watcher, disabling cache 2024-03-04T18:04:08.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.714+0000 7f0a6ee90980 20 remove_watcher() i=1 2024-03-04T18:04:08.711 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.718+0000 7f0a6ee90980 20 remove_watcher() i=2 2024-03-04T18:04:08.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.718+0000 7f0a6ee90980 20 remove_watcher() i=3 2024-03-04T18:04:08.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.722+0000 7f0a6ee90980 20 remove_watcher() i=4 2024-03-04T18:04:08.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.722+0000 7f0a6ee90980 20 remove_watcher() i=5 2024-03-04T18:04:08.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.726+0000 7f0a6ee90980 20 remove_watcher() i=6 2024-03-04T18:04:08.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:08.730+0000 7f0a6ee90980 20 remove_watcher() i=7 2024-03-04T18:04:08.739 INFO:tasks.workunit.client.0.smithi047.stderr:waiting 30 seconds for reshard lock to expire... 2024-03-04T18:04:38.771 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:38.807 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:38.807 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:38.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.857+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:38.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.857+0000 7f9d8c5dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:38.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.857+0000 7f9d8c5dc980 20 realm 2024-03-04T18:04:38.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.857+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:38.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.861+0000 7f9d8c5dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:38.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.861+0000 7f9d8c5dc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:38.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.861+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:38.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.861+0000 7f9d8c5dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:38.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.861+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:38.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.861+0000 7f9d8c5dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:38.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.861+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:38.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.861+0000 7f9d8c5dc980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:38.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.861+0000 7f9d8c5dc980 20 searching for the correct realm 2024-03-04T18:04:38.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:38.865+0000 7f9d8a0f4640 0 -- 172.21.15.47:0/1385012787 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f9d7c0f2a10 msgr2=0x7f9d7c0e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:39.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.077+0000 7f9d8a0f4640 0 -- 172.21.15.47:0/1385012787 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f9d8403e040 msgr2=0x7f9d8400f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:39.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.293+0000 7f9d8c5dc980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:39.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.293+0000 7f9d8c5dc980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:39.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.293+0000 7f9d8c5dc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:39.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:39.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:39.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:39.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:39.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:39.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 zone default found 2024-03-04T18:04:39.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 4 Realm: () 2024-03-04T18:04:39.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:39.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:39.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:39.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 zonegroup default 2024-03-04T18:04:39.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.297+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.301+0000 7f9d8c5dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.301+0000 7f9d8c5dc980 20 started sync module instance, tier type = 2024-03-04T18:04:39.296 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.301+0000 7f9d8c5dc980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:39.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.321+0000 7f9d8c5dc980 20 add_watcher() i=0 2024-03-04T18:04:39.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.321+0000 7f9d8c5dc980 20 add_watcher() i=1 2024-03-04T18:04:39.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.321+0000 7f9d8c5dc980 20 add_watcher() i=2 2024-03-04T18:04:39.318 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.321+0000 7f9d8c5dc980 20 add_watcher() i=3 2024-03-04T18:04:39.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.321+0000 7f9d8c5dc980 20 add_watcher() i=4 2024-03-04T18:04:39.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.321+0000 7f9d8c5dc980 20 add_watcher() i=5 2024-03-04T18:04:39.319 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.321+0000 7f9d8c5dc980 20 add_watcher() i=6 2024-03-04T18:04:39.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.325+0000 7f9d8c5dc980 20 add_watcher() i=7 2024-03-04T18:04:39.321 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.325+0000 7f9d8c5dc980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:39.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9c417e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:39.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9d8c5dc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:39.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9d8c5dc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:39.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9d8c5dc980 5 note: GC not initialized 2024-03-04T18:04:39.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9c3b7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:39.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9d8c5dc980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:39.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9d8c5dc980 20 Filter name: none 2024-03-04T18:04:39.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9c397fa640 20 reqs_thread_entry: start 2024-03-04T18:04:39.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9d8c5dc980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:04:39.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.329+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:39.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:04:39.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:04:39.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 : miss 2024-03-04T18:04:39.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:39.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 info.flags=0x17 2024-03-04T18:04:39.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to cache LRU end 2024-03-04T18:04:39.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:39.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:39.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.333+0000 7f9d8c5dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 2024-03-04T18:04:39.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.341+0000 7f9d8c5dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 info.flags=0x17 2024-03-04T18:04:39.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.341+0000 7f9d8c5dc980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to cache LRU end 2024-03-04T18:04:39.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.341+0000 7f9d8c5dc980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:39.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.341+0000 7f9d8c5dc980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21, ofs0, ns] 2024-03-04T18:04:39.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.341+0000 7f9c6dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595108 cookie 93906553240480 notifier 6375 bl.length()=887 2024-03-04T18:04:39.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.341+0000 7f9c6dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 info.flags=0x17 2024-03-04T18:04:39.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.341+0000 7f9c6dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to cache LRU end 2024-03-04T18:04:39.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.341+0000 7f9c6dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:39.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.345+0000 7f9d8c5dc980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.21]): orig_sources= new_sources= 2024-03-04T18:04:39.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.345+0000 7f9d8c5dc980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.21]): potential sources added=[] removed=[] 2024-03-04T18:04:39.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.345+0000 7f9d8c5dc980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.21]): orig_dests= new_dests= 2024-03-04T18:04:39.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.345+0000 7f9d8c5dc980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.21]): potential dests added=[] removed=[] 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.0 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.0 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.1 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.1 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.2 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.2 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.3 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.3 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.4 2024-03-04T18:04:39.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.4 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.5 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.5 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.6 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.6 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.7 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.7 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.8 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.8 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.9 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.9 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.10 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.10 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.11 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1.11 2024-03-04T18:04:39.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.349+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.0 2024-03-04T18:04:39.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.353+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.1 2024-03-04T18:04:39.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.353+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.2 2024-03-04T18:04:39.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.353+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.3 2024-03-04T18:04:39.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.353+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.4 2024-03-04T18:04:39.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.357+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.5 2024-03-04T18:04:39.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.357+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.6 2024-03-04T18:04:39.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.357+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.7 2024-03-04T18:04:39.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.357+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.8 2024-03-04T18:04:39.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.361+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.9 2024-03-04T18:04:39.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.361+0000 7f9d8c5dc980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.21.10 2024-03-04T18:04:39.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.369+0000 7f9d8c5dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:39.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.369+0000 7f9d8c5dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 2024-03-04T18:04:39.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.369+0000 7f9d8c5dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 info.flags=0x17 2024-03-04T18:04:39.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.369+0000 7f9d8c5dc980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to cache LRU end 2024-03-04T18:04:39.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.369+0000 7f9d8c5dc980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:39.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.369+0000 7f9d8c5dc980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21, ofs0, ns] 2024-03-04T18:04:39.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.373+0000 7f9c6d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595109 cookie 93906553240480 notifier 6375 bl.length()=855 2024-03-04T18:04:39.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.373+0000 7f9c6d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 info.flags=0x17 2024-03-04T18:04:39.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.373+0000 7f9c6d7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.21 to cache LRU end 2024-03-04T18:04:39.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.373+0000 7f9c6d7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:39.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.373+0000 7f9d8c5dc980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.21]): orig_sources= new_sources= 2024-03-04T18:04:39.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.373+0000 7f9d8c5dc980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.21]): potential sources added=[] removed=[] 2024-03-04T18:04:39.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.373+0000 7f9d8c5dc980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.21]): orig_dests= new_dests= 2024-03-04T18:04:39.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.373+0000 7f9d8c5dc980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.21]): potential dests added=[] removed=[] 2024-03-04T18:04:39.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.381+0000 7f9d8c5dc980 1 execute INFO: reshard of bucket "abort-at-set-target-layout" completed successfully 2024-03-04T18:04:39.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.385+0000 7f9c417e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:39.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.389+0000 7f9d8c5dc980 20 remove_watcher() i=0 2024-03-04T18:04:39.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.389+0000 7f9d8c5dc980 2 removed watcher, disabling cache 2024-03-04T18:04:39.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.389+0000 7f9d8c5dc980 20 remove_watcher() i=1 2024-03-04T18:04:39.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.393+0000 7f9d8c5dc980 20 remove_watcher() i=2 2024-03-04T18:04:39.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.393+0000 7f9d8c5dc980 20 remove_watcher() i=3 2024-03-04T18:04:39.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.397+0000 7f9d8c5dc980 20 remove_watcher() i=4 2024-03-04T18:04:39.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.397+0000 7f9d8c5dc980 20 remove_watcher() i=5 2024-03-04T18:04:39.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.401+0000 7f9d8c5dc980 20 remove_watcher() i=6 2024-03-04T18:04:39.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.405+0000 7f9d8c5dc980 20 remove_watcher() i=7 2024-03-04T18:04:39.416 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-03-04T18:04:39.448 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:39.448 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:39.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.497+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.497+0000 7f69ee4da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.497+0000 7f69ee4da980 20 realm 2024-03-04T18:04:39.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.497+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.497+0000 7f69ee4da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.497+0000 7f69ee4da980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:39.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.497+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.497+0000 7f69ee4da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.497+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.501+0000 7f69ee4da980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:39.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.501+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.501+0000 7f69ee4da980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:39.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.501+0000 7f69ee4da980 20 searching for the correct realm 2024-03-04T18:04:39.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.505+0000 7f69eb7f1640 0 -- 172.21.15.47:0/1436432843 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f69e003ebe0 msgr2=0x7f69e002c910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:39.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.513+0000 7f69eb7f1640 0 -- 172.21.15.47:0/1436432843 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x56525ba49310 msgr2=0x56525ba4b7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:39.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:39.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:39.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:39.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:39.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:39.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:39.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:39.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:39.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 zone default found 2024-03-04T18:04:39.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 4 Realm: () 2024-03-04T18:04:39.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:39.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:39.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:39.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 zonegroup default 2024-03-04T18:04:39.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.521+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.525+0000 7f69ee4da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.525+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.525+0000 7f69ee4da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.525+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.525+0000 7f69ee4da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.525+0000 7f69ee4da980 20 started sync module instance, tier type = 2024-03-04T18:04:39.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.525+0000 7f69ee4da980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:39.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.541+0000 7f69ee4da980 20 add_watcher() i=0 2024-03-04T18:04:39.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.541+0000 7f69ee4da980 20 add_watcher() i=1 2024-03-04T18:04:39.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.541+0000 7f69ee4da980 20 add_watcher() i=2 2024-03-04T18:04:39.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.541+0000 7f69ee4da980 20 add_watcher() i=3 2024-03-04T18:04:39.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.541+0000 7f69ee4da980 20 add_watcher() i=4 2024-03-04T18:04:39.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.541+0000 7f69ee4da980 20 add_watcher() i=5 2024-03-04T18:04:39.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.541+0000 7f69ee4da980 20 add_watcher() i=6 2024-03-04T18:04:39.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.545+0000 7f69ee4da980 20 add_watcher() i=7 2024-03-04T18:04:39.541 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.545+0000 7f69ee4da980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:39.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f68a1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:39.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:39.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:39.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 5 note: GC not initialized 2024-03-04T18:04:39.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f68a0fe1640 20 reqs_thread_entry: start 2024-03-04T18:04:39.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:39.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 Filter name: none 2024-03-04T18:04:39.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f681a7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:39.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:39.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:39.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:39.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rados_obj.operate() r=0 bl.length=194 2024-03-04T18:04:39.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:04:39.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.549+0000 7f69ee4da980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:39.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.553+0000 7f69ee4da980 20 rados->read ofs=0 len=0 2024-03-04T18:04:39.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.553+0000 7f69ee4da980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:39.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.553+0000 7f68a1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:39.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.557+0000 7f69ee4da980 20 remove_watcher() i=0 2024-03-04T18:04:39.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.557+0000 7f69ee4da980 2 removed watcher, disabling cache 2024-03-04T18:04:39.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.561+0000 7f69ee4da980 20 remove_watcher() i=1 2024-03-04T18:04:39.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.561+0000 7f69ee4da980 20 remove_watcher() i=2 2024-03-04T18:04:39.560 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.565+0000 7f69ee4da980 20 remove_watcher() i=3 2024-03-04T18:04:39.562 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.565+0000 7f69ee4da980 20 remove_watcher() i=4 2024-03-04T18:04:39.564 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.569+0000 7f69ee4da980 20 remove_watcher() i=5 2024-03-04T18:04:39.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.569+0000 7f69ee4da980 20 remove_watcher() i=6 2024-03-04T18:04:39.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:39.573+0000 7f69ee4da980 20 remove_watcher() i=7 2024-03-04T18:04:39.583 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:39.583 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:39.583 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-set-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.21 num_objects: 19 size_kb: 1 num_shards: 12 2024-03-04T18:04:39.583 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:39.584 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:39.585 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:39.605 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:39.606 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:40.017 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:40.017 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:40.408 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with EIO injected at block_writes 2024-03-04T18:04:40.408 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:40.411 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:40.412 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:40.542 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:40.543 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:40.624 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:40.624 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:40.723 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:40.723 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:40.808 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:40.808 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:40.892 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:40.892 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:40.976 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:40.976 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.064 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.064 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.150 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.150 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.234 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.234 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.322 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.322 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.405 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.405 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.494 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.494 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.578 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.578 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.667 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.667 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.736 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.736 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.821 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.821 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.906 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.906 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:41.992 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:41.992 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:42.078 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:42.079 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:42.167 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:42.167 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:42.252 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:42.252 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:42.336 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-03-04T18:04:42.370 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:42.370 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:42.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.417+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 20 realm 2024-03-04T18:04:42.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:42.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:42.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.421+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.420 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.425+0000 7f85d762e980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:42.420 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.425+0000 7f85d762e980 20 searching for the correct realm 2024-03-04T18:04:42.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.433+0000 7f85d5146640 0 -- 172.21.15.47:0/429134762 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f85c00dd060 msgr2=0x7f85c012b190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:42.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.437+0000 7f85d5146640 0 -- 172.21.15.47:0/429134762 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f85d0008040 msgr2=0x7f85d00349b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:42.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:42.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:42.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:42.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:42.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:42.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:42.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:42.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:42.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 zone default found 2024-03-04T18:04:42.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 4 Realm: () 2024-03-04T18:04:42.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:42.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:42.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:42.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 zonegroup default 2024-03-04T18:04:42.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.445+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.449+0000 7f85d762e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.449+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.449+0000 7f85d762e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.449+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.449+0000 7f85d762e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.449+0000 7f85d762e980 20 started sync module instance, tier type = 2024-03-04T18:04:42.445 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.449+0000 7f85d762e980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:42.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.465+0000 7f85d762e980 20 add_watcher() i=0 2024-03-04T18:04:42.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.465+0000 7f85d762e980 20 add_watcher() i=1 2024-03-04T18:04:42.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.465+0000 7f85d762e980 20 add_watcher() i=2 2024-03-04T18:04:42.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.465+0000 7f85d762e980 20 add_watcher() i=3 2024-03-04T18:04:42.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.465+0000 7f85d762e980 20 add_watcher() i=4 2024-03-04T18:04:42.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.465+0000 7f85d762e980 20 add_watcher() i=5 2024-03-04T18:04:42.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.465+0000 7f85d762e980 20 add_watcher() i=6 2024-03-04T18:04:42.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.469+0000 7f85d762e980 20 add_watcher() i=7 2024-03-04T18:04:42.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.469+0000 7f85d762e980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:42.469 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f848b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:42.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f85d762e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:42.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f85d762e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:42.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f85d762e980 5 note: GC not initialized 2024-03-04T18:04:42.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f848a7e4640 20 reqs_thread_entry: start 2024-03-04T18:04:42.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f85d762e980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:42.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f85d762e980 20 Filter name: none 2024-03-04T18:04:42.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f83fbfff640 20 reqs_thread_entry: start 2024-03-04T18:04:42.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f85d762e980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:42.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.473+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.477+0000 7f85d762e980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:42.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.477+0000 7f85d762e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:42.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.477+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.477+0000 7f85d762e980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:42.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.477+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.477+0000 7f85d762e980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:42.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.477+0000 7f85d762e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:42.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.477+0000 7f85d762e980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.477+0000 7f85d762e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.481+0000 7f848b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:42.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.485+0000 7f85d762e980 20 remove_watcher() i=0 2024-03-04T18:04:42.480 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.485+0000 7f85d762e980 2 removed watcher, disabling cache 2024-03-04T18:04:42.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.485+0000 7f85d762e980 20 remove_watcher() i=1 2024-03-04T18:04:42.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.489+0000 7f85d762e980 20 remove_watcher() i=2 2024-03-04T18:04:42.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.489+0000 7f85d762e980 20 remove_watcher() i=3 2024-03-04T18:04:42.488 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.493+0000 7f85d762e980 20 remove_watcher() i=4 2024-03-04T18:04:42.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.493+0000 7f85d762e980 20 remove_watcher() i=5 2024-03-04T18:04:42.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.497+0000 7f85d762e980 20 remove_watcher() i=6 2024-03-04T18:04:42.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.501+0000 7f85d762e980 20 remove_watcher() i=7 2024-03-04T18:04:42.509 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:42.509 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:42.509 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-block-writes id: 797b67a4-b876-4d04-b122-10afd858d465.4453.22 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:04:42.509 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:42.510 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at block_writes 2024-03-04T18:04:42.540 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:42.541 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:42.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.589+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.589+0000 7f7050912980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.589+0000 7f7050912980 20 realm 2024-03-04T18:04:42.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.589+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.589+0000 7f7050912980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.589+0000 7f7050912980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:42.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.589+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.593+0000 7f7050912980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.593+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.593+0000 7f7050912980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:42.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.593+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.593+0000 7f7050912980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:42.590 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.593+0000 7f7050912980 20 searching for the correct realm 2024-03-04T18:04:42.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.601+0000 7f704dc29640 0 -- 172.21.15.47:0/2752494749 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f7040103210 msgr2=0x7f70400f32d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:42.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.605+0000 7f704ec2b640 0 -- 172.21.15.47:0/2752494749 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x565084536f70 msgr2=0x5650845393f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:42.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.613+0000 7f7050912980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:42.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.613+0000 7f7050912980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:42.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.613+0000 7f7050912980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:42.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.613+0000 7f7050912980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:42.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.613+0000 7f7050912980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:42.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.613+0000 7f7050912980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:42.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.613+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 zone default found 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 4 Realm: () 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 zonegroup default 2024-03-04T18:04:42.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 started sync module instance, tier type = 2024-03-04T18:04:42.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.617+0000 7f7050912980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:42.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.637+0000 7f7050912980 20 add_watcher() i=0 2024-03-04T18:04:42.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.637+0000 7f7050912980 20 add_watcher() i=1 2024-03-04T18:04:42.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.637+0000 7f7050912980 20 add_watcher() i=2 2024-03-04T18:04:42.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.637+0000 7f7050912980 20 add_watcher() i=3 2024-03-04T18:04:42.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.637+0000 7f7050912980 20 add_watcher() i=4 2024-03-04T18:04:42.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.637+0000 7f7050912980 20 add_watcher() i=5 2024-03-04T18:04:42.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.637+0000 7f7050912980 20 add_watcher() i=6 2024-03-04T18:04:42.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.637+0000 7f7050912980 20 add_watcher() i=7 2024-03-04T18:04:42.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.637+0000 7f7050912980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:42.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f6f04fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:42.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:42.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:42.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 5 note: GC not initialized 2024-03-04T18:04:42.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f6e7f7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:42.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:42.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 20 Filter name: none 2024-03-04T18:04:42.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f6e7d7fa640 20 reqs_thread_entry: start 2024-03-04T18:04:42.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-03-04T18:04:42.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 : miss 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.641+0000 7f7050912980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.645+0000 7f7050912980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.645+0000 7f7050912980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.645+0000 7f7050912980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.645+0000 7f7050912980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.645+0000 7f7050912980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:42.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.645+0000 7f7050912980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 2024-03-04T18:04:42.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.653+0000 7f7050912980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:42.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.653+0000 7f7050912980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:42.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.653+0000 7f7050912980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:42.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.653+0000 7f7050912980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22, ofs0, ns] 2024-03-04T18:04:42.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.653+0000 7f6f317fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595090 cookie 94903817440960 notifier 6062 bl.length()=877 2024-03-04T18:04:42.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.653+0000 7f6f317fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:42.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.653+0000 7f6f317fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:42.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.653+0000 7f6f317fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:42.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.657+0000 7f7050912980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): orig_sources= new_sources= 2024-03-04T18:04:42.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.657+0000 7f7050912980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): potential sources added=[] removed=[] 2024-03-04T18:04:42.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.657+0000 7f7050912980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): orig_dests= new_dests= 2024-03-04T18:04:42.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.657+0000 7f7050912980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): potential dests added=[] removed=[] 2024-03-04T18:04:42.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.657+0000 7f7050912980 -1 Injecting error=-5 at location=block_writes 2024-03-04T18:04:42.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.657+0000 7f7050912980 0 ERROR: init_reshard failed to pause writes to the current index: (5) Input/output error 2024-03-04T18:04:42.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.661+0000 7f7050912980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:42.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.661+0000 7f7050912980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 2024-03-04T18:04:42.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f7050912980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:42.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f7050912980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:42.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f7050912980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:42.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f7050912980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22, ofs0, ns] 2024-03-04T18:04:42.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f6f31ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595091 cookie 94903817440960 notifier 6062 bl.length()=845 2024-03-04T18:04:42.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f6f31ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:42.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f6f31ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:42.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f6f31ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:42.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f7050912980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): orig_sources= new_sources= 2024-03-04T18:04:42.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f7050912980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): potential sources added=[] removed=[] 2024-03-04T18:04:42.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f7050912980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): orig_dests= new_dests= 2024-03-04T18:04:42.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.665+0000 7f7050912980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): potential dests added=[] removed=[] 2024-03-04T18:04:42.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.669+0000 7f6f04fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:42.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.673+0000 7f7050912980 20 remove_watcher() i=0 2024-03-04T18:04:42.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.673+0000 7f7050912980 2 removed watcher, disabling cache 2024-03-04T18:04:42.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.677+0000 7f7050912980 20 remove_watcher() i=1 2024-03-04T18:04:42.674 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.677+0000 7f7050912980 20 remove_watcher() i=2 2024-03-04T18:04:42.675 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.681+0000 7f7050912980 20 remove_watcher() i=3 2024-03-04T18:04:42.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.681+0000 7f7050912980 20 remove_watcher() i=4 2024-03-04T18:04:42.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.685+0000 7f7050912980 20 remove_watcher() i=5 2024-03-04T18:04:42.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.685+0000 7f7050912980 20 remove_watcher() i=6 2024-03-04T18:04:42.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.689+0000 7f7050912980 20 remove_watcher() i=7 2024-03-04T18:04:42.699 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-03-04T18:04:42.725 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:42.725 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:42.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.773+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.773+0000 7f9cebd27980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.773+0000 7f9cebd27980 20 realm 2024-03-04T18:04:42.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.773+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.773+0000 7f9cebd27980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.773+0000 7f9cebd27980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:42.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.773+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.777+0000 7f9cebd27980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:42.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.777+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.777+0000 7f9cebd27980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:42.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.777+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:42.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.777+0000 7f9cebd27980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:42.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.777+0000 7f9cebd27980 20 searching for the correct realm 2024-03-04T18:04:42.780 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.785+0000 7f9ce903e640 0 -- 172.21.15.47:0/3351555796 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f9ce0037410 msgr2=0x7f9ce0037890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:42.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:42.997+0000 7f9cea040640 0 -- 172.21.15.47:0/3351555796 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55f47479a1f0 msgr2=0x55f47479c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:43.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:43.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:43.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:43.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:43.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:43.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:43.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:43.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:43.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 zone default found 2024-03-04T18:04:43.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 4 Realm: () 2024-03-04T18:04:43.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:43.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:43.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:43.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 zonegroup default 2024-03-04T18:04:43.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.011 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.013+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.017+0000 7f9cebd27980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.017+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.017+0000 7f9cebd27980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.017+0000 7f9cebd27980 20 started sync module instance, tier type = 2024-03-04T18:04:43.013 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.017+0000 7f9cebd27980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:43.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.041+0000 7f9cebd27980 20 add_watcher() i=0 2024-03-04T18:04:43.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.041+0000 7f9cebd27980 20 add_watcher() i=1 2024-03-04T18:04:43.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.041+0000 7f9cebd27980 20 add_watcher() i=2 2024-03-04T18:04:43.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.041+0000 7f9cebd27980 20 add_watcher() i=3 2024-03-04T18:04:43.037 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.041+0000 7f9cebd27980 20 add_watcher() i=4 2024-03-04T18:04:43.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.041+0000 7f9cebd27980 20 add_watcher() i=5 2024-03-04T18:04:43.038 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.041+0000 7f9cebd27980 20 add_watcher() i=6 2024-03-04T18:04:43.040 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.045+0000 7f9cebd27980 20 add_watcher() i=7 2024-03-04T18:04:43.040 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.045+0000 7f9cebd27980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:43.044 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.049+0000 7f9b9b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.049+0000 7f9cebd27980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:43.045 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.049+0000 7f9cebd27980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:43.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.049+0000 7f9cebd27980 5 note: GC not initialized 2024-03-04T18:04:43.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.049+0000 7f9b9a7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:43.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.049+0000 7f9cebd27980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:43.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.049+0000 7f9cebd27980 20 Filter name: none 2024-03-04T18:04:43.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.049+0000 7f9b0ffff640 20 reqs_thread_entry: start 2024-03-04T18:04:43.047 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.049+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.053+0000 7f9cebd27980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:43.048 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.053+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.053+0000 7f9cebd27980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:43.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.053+0000 7f9cebd27980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:43.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.053+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.053+0000 7f9cebd27980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:43.049 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.053+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.050 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.053+0000 7f9cebd27980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:43.050 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.053+0000 7f9cebd27980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:43.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.057+0000 7f9cebd27980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.057+0000 7f9cebd27980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.057+0000 7f9b9b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.061+0000 7f9cebd27980 20 remove_watcher() i=0 2024-03-04T18:04:43.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.061+0000 7f9cebd27980 2 removed watcher, disabling cache 2024-03-04T18:04:43.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.065+0000 7f9cebd27980 20 remove_watcher() i=1 2024-03-04T18:04:43.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.065+0000 7f9cebd27980 20 remove_watcher() i=2 2024-03-04T18:04:43.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.069+0000 7f9cebd27980 20 remove_watcher() i=3 2024-03-04T18:04:43.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.073+0000 7f9cebd27980 20 remove_watcher() i=4 2024-03-04T18:04:43.070 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.073+0000 7f9cebd27980 20 remove_watcher() i=5 2024-03-04T18:04:43.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.077+0000 7f9cebd27980 20 remove_watcher() i=6 2024-03-04T18:04:43.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.081+0000 7f9cebd27980 20 remove_watcher() i=7 2024-03-04T18:04:43.091 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:43.092 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:43.092 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-block-writes id: 797b67a4-b876-4d04-b122-10afd858d465.4453.22 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:04:43.092 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:43.093 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:43.093 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:43.134 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:43.134 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:43.194 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:43.231 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:43.231 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:43.275 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.277+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.281+0000 7fd1cd64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.281+0000 7fd1cd64d980 20 realm 2024-03-04T18:04:43.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.281+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.281+0000 7fd1cd64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.281+0000 7fd1cd64d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:43.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.281+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.281+0000 7fd1cd64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.281+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.285+0000 7fd1cd64d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:43.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.285+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.285+0000 7fd1cd64d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:43.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.285+0000 7fd1cd64d980 20 searching for the correct realm 2024-03-04T18:04:43.288 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.293+0000 7fd1ca964640 0 -- 172.21.15.47:0/1906594644 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fd1c00f8df0 msgr2=0x7fd1c00c28f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:43.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.297+0000 7fd1cb966640 0 -- 172.21.15.47:0/1906594644 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x558ca0138ee0 msgr2=0x558ca013b370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:43.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:43.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:43.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:43.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:43.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:43.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:43.302 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:43.303 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 zone default found 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 4 Realm: () 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 zonegroup default 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.305+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.309+0000 7fd1cd64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.304 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.309+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.309+0000 7fd1cd64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.309+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.309+0000 7fd1cd64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.309+0000 7fd1cd64d980 20 started sync module instance, tier type = 2024-03-04T18:04:43.305 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.309+0000 7fd1cd64d980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:43.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.325+0000 7fd1cd64d980 20 add_watcher() i=0 2024-03-04T18:04:43.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.325+0000 7fd1cd64d980 20 add_watcher() i=1 2024-03-04T18:04:43.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.325+0000 7fd1cd64d980 20 add_watcher() i=2 2024-03-04T18:04:43.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.325+0000 7fd1cd64d980 20 add_watcher() i=3 2024-03-04T18:04:43.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.325+0000 7fd1cd64d980 20 add_watcher() i=4 2024-03-04T18:04:43.323 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.325+0000 7fd1cd64d980 20 add_watcher() i=5 2024-03-04T18:04:43.324 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.325+0000 7fd1cd64d980 20 add_watcher() i=6 2024-03-04T18:04:43.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.329+0000 7fd1cd64d980 20 add_watcher() i=7 2024-03-04T18:04:43.326 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.329+0000 7fd1cd64d980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:43.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd0827e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:43.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:43.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 5 note: GC not initialized 2024-03-04T18:04:43.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd0817e2640 20 reqs_thread_entry: start 2024-03-04T18:04:43.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:43.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 20 Filter name: none 2024-03-04T18:04:43.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd07a7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:43.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-03-04T18:04:43.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:43.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-03-04T18:04:43.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-03-04T18:04:43.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 : miss 2024-03-04T18:04:43.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:43.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:43.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:43.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:43.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:43.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.333+0000 7fd1cd64d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 2024-03-04T18:04:43.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.345+0000 7fd1cd64d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:43.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.345+0000 7fd1cd64d980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:43.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.345+0000 7fd1cd64d980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:43.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.345+0000 7fd1cd64d980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22, ofs0, ns] 2024-03-04T18:04:43.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.345+0000 7fd0ae7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595092 cookie 94062469426752 notifier 6468 bl.length()=877 2024-03-04T18:04:43.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.345+0000 7fd0ae7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:43.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.345+0000 7fd0ae7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:43.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.345+0000 7fd0ae7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:43.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.349+0000 7fd1cd64d980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): orig_sources= new_sources= 2024-03-04T18:04:43.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.349+0000 7fd1cd64d980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): potential sources added=[] removed=[] 2024-03-04T18:04:43.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.349+0000 7fd1cd64d980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): orig_dests= new_dests= 2024-03-04T18:04:43.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.349+0000 7fd1cd64d980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): potential dests added=[] removed=[] 2024-03-04T18:04:43.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.0 2024-03-04T18:04:43.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.0 2024-03-04T18:04:43.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.1 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.1 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.2 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.2 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.3 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.3 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.4 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.4 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.5 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.5 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.6 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.6 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.7 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.7 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.8 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.8 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.9 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.9 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.10 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.10 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.11 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1.11 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.0 2024-03-04T18:04:43.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.1 2024-03-04T18:04:43.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.353+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.2 2024-03-04T18:04:43.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.357+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.3 2024-03-04T18:04:43.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.357+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.4 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.357+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.5 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.357+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.6 2024-03-04T18:04:43.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.357+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.7 2024-03-04T18:04:43.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.357+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.8 2024-03-04T18:04:43.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.361+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.9 2024-03-04T18:04:43.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.361+0000 7fd1cd64d980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.22.10 2024-03-04T18:04:43.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.365+0000 7fd1cd64d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:43.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.365+0000 7fd1cd64d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 2024-03-04T18:04:43.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd1cd64d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:43.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd1cd64d980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:43.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd1cd64d980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:43.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd1cd64d980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22, ofs0, ns] 2024-03-04T18:04:43.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd0ae7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595093 cookie 94062469426752 notifier 6468 bl.length()=845 2024-03-04T18:04:43.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd0ae7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 info.flags=0x17 2024-03-04T18:04:43.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd0ae7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.22 to cache LRU end 2024-03-04T18:04:43.366 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd0ae7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:43.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd1cd64d980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): orig_sources= new_sources= 2024-03-04T18:04:43.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd1cd64d980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): potential sources added=[] removed=[] 2024-03-04T18:04:43.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd1cd64d980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): orig_dests= new_dests= 2024-03-04T18:04:43.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.369+0000 7fd1cd64d980 20 handle_bi_update(): bucket=:error-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.22]): potential dests added=[] removed=[] 2024-03-04T18:04:43.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.377+0000 7fd1cd64d980 1 execute INFO: reshard of bucket "error-at-block-writes" completed successfully 2024-03-04T18:04:43.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.377+0000 7fd0827e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.381+0000 7fd1cd64d980 20 remove_watcher() i=0 2024-03-04T18:04:43.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.381+0000 7fd1cd64d980 2 removed watcher, disabling cache 2024-03-04T18:04:43.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.385+0000 7fd1cd64d980 20 remove_watcher() i=1 2024-03-04T18:04:43.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.385+0000 7fd1cd64d980 20 remove_watcher() i=2 2024-03-04T18:04:43.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.389+0000 7fd1cd64d980 20 remove_watcher() i=3 2024-03-04T18:04:43.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.389+0000 7fd1cd64d980 20 remove_watcher() i=4 2024-03-04T18:04:43.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.393+0000 7fd1cd64d980 20 remove_watcher() i=5 2024-03-04T18:04:43.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.397+0000 7fd1cd64d980 20 remove_watcher() i=6 2024-03-04T18:04:43.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.397+0000 7fd1cd64d980 20 remove_watcher() i=7 2024-03-04T18:04:43.411 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-03-04T18:04:43.438 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:43.438 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:43.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.485+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 realm 2024-03-04T18:04:43.484 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:43.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.485 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:43.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:43.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.489+0000 7fc11a4b3980 20 searching for the correct realm 2024-03-04T18:04:43.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.497+0000 7fc1177ca640 0 -- 172.21.15.47:0/3566834491 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fc10c127550 msgr2=0x7fc10c1265f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:43.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.709+0000 7fc1187cc640 0 -- 172.21.15.47:0/3566834491 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55a148648540 msgr2=0x55a14864a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:43.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.721+0000 7fc11a4b3980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:43.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.721+0000 7fc11a4b3980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:43.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.721+0000 7fc11a4b3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:43.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.721+0000 7fc11a4b3980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:43.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.721+0000 7fc11a4b3980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:43.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.721+0000 7fc11a4b3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:43.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.721+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:43.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:43.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 zone default found 2024-03-04T18:04:43.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 4 Realm: () 2024-03-04T18:04:43.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:43.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:43.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:43.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 zonegroup default 2024-03-04T18:04:43.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.725+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.729+0000 7fc11a4b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.729+0000 7fc11a4b3980 20 started sync module instance, tier type = 2024-03-04T18:04:43.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.729+0000 7fc11a4b3980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:43.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.753+0000 7fc11a4b3980 20 add_watcher() i=0 2024-03-04T18:04:43.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.753+0000 7fc11a4b3980 20 add_watcher() i=1 2024-03-04T18:04:43.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.753+0000 7fc11a4b3980 20 add_watcher() i=2 2024-03-04T18:04:43.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.753+0000 7fc11a4b3980 20 add_watcher() i=3 2024-03-04T18:04:43.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.753+0000 7fc11a4b3980 20 add_watcher() i=4 2024-03-04T18:04:43.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.753+0000 7fc11a4b3980 20 add_watcher() i=5 2024-03-04T18:04:43.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.753+0000 7fc11a4b3980 20 add_watcher() i=6 2024-03-04T18:04:43.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.753+0000 7fc11a4b3980 20 add_watcher() i=7 2024-03-04T18:04:43.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.753+0000 7fc11a4b3980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:43.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fbfcdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fc11a4b3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:43.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fc11a4b3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:43.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fc11a4b3980 5 note: GC not initialized 2024-03-04T18:04:43.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fbfccfe1640 20 reqs_thread_entry: start 2024-03-04T18:04:43.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fc11a4b3980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:43.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fc11a4b3980 20 Filter name: none 2024-03-04T18:04:43.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fbf467fc640 20 reqs_thread_entry: start 2024-03-04T18:04:43.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fc11a4b3980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:43.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.761+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.765+0000 7fc11a4b3980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:43.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.765+0000 7fc11a4b3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:43.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.765+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.765+0000 7fc11a4b3980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:43.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.765+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.765+0000 7fc11a4b3980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:43.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.765+0000 7fc11a4b3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:43.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.769+0000 7fc11a4b3980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.769+0000 7fc11a4b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.769+0000 7fbfcdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.777+0000 7fc11a4b3980 20 remove_watcher() i=0 2024-03-04T18:04:43.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.777+0000 7fc11a4b3980 2 removed watcher, disabling cache 2024-03-04T18:04:43.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.777+0000 7fc11a4b3980 20 remove_watcher() i=1 2024-03-04T18:04:43.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.781+0000 7fc11a4b3980 20 remove_watcher() i=2 2024-03-04T18:04:43.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.781+0000 7fc11a4b3980 20 remove_watcher() i=3 2024-03-04T18:04:43.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.785+0000 7fc11a4b3980 20 remove_watcher() i=4 2024-03-04T18:04:43.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.789+0000 7fc11a4b3980 20 remove_watcher() i=5 2024-03-04T18:04:43.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.793+0000 7fc11a4b3980 20 remove_watcher() i=6 2024-03-04T18:04:43.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:43.793+0000 7fc11a4b3980 20 remove_watcher() i=7 2024-03-04T18:04:43.811 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:43.811 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:43.811 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-block-writes id: 797b67a4-b876-4d04-b122-10afd858d465.4453.22 num_objects: 19 size_kb: 1 num_shards: 12 2024-03-04T18:04:43.811 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:43.813 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:43.813 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:43.845 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:43.846 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:44.304 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.304 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:44.690 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with abort at block_writes 2024-03-04T18:04:44.690 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:44.693 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.693 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:44.824 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.824 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:44.908 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.908 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:44.998 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.998 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.092 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.092 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.176 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.176 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.262 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.262 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.347 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.348 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.434 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.434 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.520 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.520 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.606 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.606 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.711 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.711 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.792 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.792 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.878 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.878 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:45.962 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.962 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:46.049 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.049 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:46.137 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.138 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:46.224 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.225 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:46.310 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.310 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:46.397 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.397 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:46.481 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.481 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:46.565 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.565 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:46.647 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-03-04T18:04:46.681 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:46.682 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:46.726 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.729+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.729 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.729 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 20 realm 2024-03-04T18:04:46.729 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:46.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:46.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.733+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.737+0000 7fa709d73980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:46.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.737+0000 7fa709d73980 20 searching for the correct realm 2024-03-04T18:04:46.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.745+0000 7fa70788b640 0 -- 172.21.15.47:0/3405325146 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fa6f8018070 msgr2=0x7fa6f806c0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:46.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.957+0000 7fa70788b640 0 -- 172.21.15.47:0/3405325146 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fa7000cb070 msgr2=0x7fa7000f0fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:46.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:46.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:46.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:46.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:46.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:46.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:46.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:46.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:46.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 zone default found 2024-03-04T18:04:46.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 4 Realm: () 2024-03-04T18:04:46.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:46.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:46.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:46.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 zonegroup default 2024-03-04T18:04:46.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.969+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.973+0000 7fa709d73980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.973+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.973+0000 7fa709d73980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.973+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.973+0000 7fa709d73980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.973+0000 7fa709d73980 20 started sync module instance, tier type = 2024-03-04T18:04:46.970 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.973+0000 7fa709d73980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:46.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.997+0000 7fa709d73980 20 add_watcher() i=0 2024-03-04T18:04:46.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.997+0000 7fa709d73980 20 add_watcher() i=1 2024-03-04T18:04:46.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.997+0000 7fa709d73980 20 add_watcher() i=2 2024-03-04T18:04:46.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.997+0000 7fa709d73980 20 add_watcher() i=3 2024-03-04T18:04:46.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.997+0000 7fa709d73980 20 add_watcher() i=4 2024-03-04T18:04:46.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.997+0000 7fa709d73980 20 add_watcher() i=5 2024-03-04T18:04:46.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.997+0000 7fa709d73980 20 add_watcher() i=6 2024-03-04T18:04:46.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.997+0000 7fa709d73980 20 add_watcher() i=7 2024-03-04T18:04:46.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:46.997+0000 7fa709d73980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:46.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.001+0000 7fa5bdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa709d73980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:47.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa709d73980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:47.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa709d73980 5 note: GC not initialized 2024-03-04T18:04:47.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa5bcfe1640 20 reqs_thread_entry: start 2024-03-04T18:04:47.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa709d73980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:47.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa709d73980 20 Filter name: none 2024-03-04T18:04:47.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa5367fc640 20 reqs_thread_entry: start 2024-03-04T18:04:47.002 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.003 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa709d73980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:47.003 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.005+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.009+0000 7fa709d73980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:47.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.009+0000 7fa709d73980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:47.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.009+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.009+0000 7fa709d73980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:47.005 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.009+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.007 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.009+0000 7fa709d73980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:47.007 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.009+0000 7fa709d73980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:47.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.013+0000 7fa709d73980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.013+0000 7fa709d73980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.013+0000 7fa5bdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.017+0000 7fa709d73980 20 remove_watcher() i=0 2024-03-04T18:04:47.015 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.017+0000 7fa709d73980 2 removed watcher, disabling cache 2024-03-04T18:04:47.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.021+0000 7fa709d73980 20 remove_watcher() i=1 2024-03-04T18:04:47.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.021+0000 7fa709d73980 20 remove_watcher() i=2 2024-03-04T18:04:47.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.025+0000 7fa709d73980 20 remove_watcher() i=3 2024-03-04T18:04:47.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.025+0000 7fa709d73980 20 remove_watcher() i=4 2024-03-04T18:04:47.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.029+0000 7fa709d73980 20 remove_watcher() i=5 2024-03-04T18:04:47.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.033+0000 7fa709d73980 20 remove_watcher() i=6 2024-03-04T18:04:47.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.037+0000 7fa709d73980 20 remove_watcher() i=7 2024-03-04T18:04:47.048 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-block-writes id: 797b67a4-b876-4d04-b122-10afd858d465.4453.23 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:04:47.048 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.048 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at block_writes 2024-03-04T18:04:47.080 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:47.081 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:47.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 realm 2024-03-04T18:04:47.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.129+0000 7fc7bcacf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:47.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 searching for the correct realm 2024-03-04T18:04:47.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.141+0000 7fc7bade8640 0 -- 172.21.15.47:0/2913834822 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55a4cc847ec0 msgr2=0x55a4cc84a310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:47.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.145+0000 7fc7ba5e7640 0 -- 172.21.15.47:0/2913834822 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x55a4cc84c110 msgr2=0x55a4cc84e540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:47.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:47.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:47.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:47.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:47.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:47.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 zone default found 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 4 Realm: () 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 zonegroup default 2024-03-04T18:04:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 started sync module instance, tier type = 2024-03-04T18:04:47.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:47.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=0 2024-03-04T18:04:47.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=1 2024-03-04T18:04:47.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=2 2024-03-04T18:04:47.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=3 2024-03-04T18:04:47.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=4 2024-03-04T18:04:47.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=5 2024-03-04T18:04:47.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=6 2024-03-04T18:04:47.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.177+0000 7fc7bcacf980 20 add_watcher() i=7 2024-03-04T18:04:47.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.177+0000 7fc7bcacf980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:47.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc670fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:47.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:47.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 note: GC not initialized 2024-03-04T18:04:47.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc5eb7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:47.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:47.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 Filter name: none 2024-03-04T18:04:47.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc5e97fa640 20 reqs_thread_entry: start 2024-03-04T18:04:47.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:04:47.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:47.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:04:47.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:04:47.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 : miss 2024-03-04T18:04:47.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.185+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:47.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:47.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 2024-03-04T18:04:47.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.192 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23, ofs0, ns] 2024-03-04T18:04:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595112 cookie 94166294289264 notifier 6146 bl.length()=877 2024-03-04T18:04:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.193 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_sources= new_sources= 2024-03-04T18:04:47.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential sources added=[] removed=[] 2024-03-04T18:04:47.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_dests= new_dests= 2024-03-04T18:04:47.195 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential dests added=[] removed=[] 2024-03-04T18:04:47.195 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fc7bcacf980 time 2024-03-04T18:04:47.203340+0000 2024-03-04T18:04:47.195 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:47.196 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fc7c03a4d7b] 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fc7c03a4f2f] 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a4cb00c281] 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x55a4cb018ced] 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc7bf20fd90] 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr:*** Caught signal (Aborted) ** 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7fc7bcacf980 thread_name:radosgw-admin 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.201+0000 7fc7bcacf980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fc7bcacf980 time 2024-03-04T18:04:47.203340+0000 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.197 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:47.198 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fc7c03a4d7b] 2024-03-04T18:04:47.198 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fc7c03a4f2f] 2024-03-04T18:04:47.198 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a4cb00c281] 2024-03-04T18:04:47.198 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x55a4cb018ced] 2024-03-04T18:04:47.198 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:04:47.198 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc7bf20fd90] 2024-03-04T18:04:47.198 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:04:47.198 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:04:47.198 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.214 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:47.214 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fc7bf228520] 2024-03-04T18:04:47.214 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:04:47.214 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:04:47.214 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:04:47.214 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fc7c03a4dd5] 2024-03-04T18:04:47.214 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fc7c03a4f2f] 2024-03-04T18:04:47.214 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a4cb00c281] 2024-03-04T18:04:47.214 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x55a4cb018ced] 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc7bf20fd90] 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.201+0000 7fc7bcacf980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7fc7bcacf980 thread_name:radosgw-admin 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fc7bf228520] 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:04:47.215 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fc7c03a4dd5] 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fc7c03a4f2f] 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a4cb00c281] 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x55a4cb018ced] 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc7bf20fd90] 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr:--- begin dump of recent events --- 2024-03-04T18:04:47.216 INFO:tasks.workunit.client.0.smithi047.stderr: -499> 2024-03-04T18:04:47.105+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr: -498> 2024-03-04T18:04:47.105+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr: -497> 2024-03-04T18:04:47.105+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_done global_id 6510 payload 274 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr: -496> 2024-03-04T18:04:47.105+0000 7fc7b9de6640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr: -495> 2024-03-04T18:04:47.105+0000 7fc7b9de6640 1 monclient: found mon.noname-a 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr: -494> 2024-03-04T18:04:47.105+0000 7fc7b9de6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr: -493> 2024-03-04T18:04:47.105+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x55a4cc61b240 auth_method 0 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr: -492> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr: -491> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr: -490> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: dump: 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.217 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr: -489> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr: -488> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr: -487> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr: -486> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr: -485> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr: -484> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: dump: 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.218 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr: -483> 2024-03-04T18:04:47.105+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6510 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr: -482> 2024-03-04T18:04:47.105+0000 7fc7bcacf980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr: -481> 2024-03-04T18:04:47.105+0000 7fc7bcacf980 4 set_mon_vals no callback set 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr: -480> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: shutdown 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr: -479> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 asok(0x55a4cc578130) unregister_commands rotate-key 2024-03-04T18:04:47.219 INFO:tasks.workunit.client.0.smithi047.stderr: -478> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: build_initial_monmap 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr: -477> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 build_initial for_mkfs: 0 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr: -476> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: monmap: 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr: -475> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding auth protocol: cephx 2024-03-04T18:04:47.220 INFO:tasks.workunit.client.0.smithi047.stderr: -474> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding auth protocol: cephx 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -473> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding auth protocol: cephx 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -472> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -471> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -470> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -469> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -468> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -467> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -466> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -465> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -464> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -463> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -462> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -461> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -460> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.221 INFO:tasks.workunit.client.0.smithi047.stderr: -459> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: starting msgr at 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -458> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: starting objecter 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -457> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command objecter_requests hook 0x55a4cc522260 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -456> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: setting wanted keys 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -455> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: calling monclient init 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -454> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: init 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -453> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding auth protocol: cephx 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -452> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding auth protocol: cephx 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -451> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding auth protocol: cephx 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -450> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -449> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -448> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -447> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -446> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.222 INFO:tasks.workunit.client.0.smithi047.stderr: -445> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -444> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -443> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -442> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -441> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -440> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -439> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -438> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -437> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -436> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command rotate-key hook 0x55a4cc75d748 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -435> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -434> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _add_conns ranks=[2,1,0] 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -433> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-c con 0x55a4cc61b240 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:47.223 INFO:tasks.workunit.client.0.smithi047.stderr: -432> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-b con 0x55a4cc760630 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -431> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-a con 0x55a4cc78d4b0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -430> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -429> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -428> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -427> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -426> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): authenticate will time out at 2458.285775s 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -425> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): get_auth_request con 0x55a4cc78d4b0 auth_method 0 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -424> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -423> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -422> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -421> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request con 0x55a4cc61b240 auth_method 0 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -420> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.224 INFO:tasks.workunit.client.0.smithi047.stderr: -419> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -418> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -417> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -416> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -415> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -414> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -413> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -412> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -411> 2024-03-04T18:04:47.109+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request con 0x55a4cc760630 auth_method 0 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -410> 2024-03-04T18:04:47.109+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -409> 2024-03-04T18:04:47.109+0000 7fc7b9de6640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -408> 2024-03-04T18:04:47.109+0000 7fc7b9de6640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -407> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): handle_auth_done global_id 6513 payload 824 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -406> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.225 INFO:tasks.workunit.client.0.smithi047.stderr: -405> 2024-03-04T18:04:47.109+0000 7fc7bade8640 1 monclient: found mon.noname-a 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr: -404> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr: -403> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr: -402> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr: -401> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: dump: 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.226 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr: -400> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr: -399> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr: -398> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr: -397> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr: -396> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr: -395> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: dump: 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.227 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -394> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6513 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -393> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -392> 2024-03-04T18:04:47.109+0000 7fc6a8608640 4 set_mon_vals no callback set 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -391> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -390> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -389> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -388> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: init done 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -387> 2024-03-04T18:04:47.113+0000 7fc6a6e05640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -386> 2024-03-04T18:04:47.113+0000 7fc6a6e05640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -385> 2024-03-04T18:04:47.113+0000 7fc6a6e05640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.228 INFO:tasks.workunit.client.0.smithi047.stderr: -384> 2024-03-04T18:04:47.113+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x7fc6a0065970 auth_method 0 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr: -383> 2024-03-04T18:04:47.113+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x55a4cc79bd80 auth_method 0 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr: -382> 2024-03-04T18:04:47.113+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x55a4cc7a8ab0 auth_method 0 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr: -381> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x55a4cc7adb40 auth_method 0 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr: -380> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: build_initial_monmap 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr: -379> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 1 build_initial for_mkfs: 0 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr: -378> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: monmap: 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:47.229 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -377> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding auth protocol: cephx 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -376> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding auth protocol: cephx 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -375> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding auth protocol: cephx 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -374> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -373> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -372> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -371> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -370> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -369> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -368> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.230 INFO:tasks.workunit.client.0.smithi047.stderr: -367> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -366> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -365> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -364> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -363> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -362> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -361> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command objecter_requests cmddesc objecter_requests hook 0x55a4cc7b6f60 EEXIST 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -360> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: init 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -359> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding auth protocol: cephx 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -358> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding auth protocol: cephx 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -357> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding auth protocol: cephx 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -356> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -355> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -354> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -353> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.231 INFO:tasks.workunit.client.0.smithi047.stderr: -352> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -351> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -350> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -349> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -348> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -347> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -346> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -345> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -344> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -343> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -342> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command rotate-key cmddesc rotate-key hook 0x55a4cc7b09d0 EEXIST 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -341> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -340> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:04:47.232 INFO:tasks.workunit.client.0.smithi047.stderr: -339> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-b con 0x55a4cc7dbbc0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -338> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-c con 0x55a4cc7dbfa0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -337> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-a con 0x55a4cc7d7160 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -336> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -335> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -334> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -333> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -332> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): authenticate will time out at 2458.295029s 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -331> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request con 0x55a4cc7dbfa0 auth_method 0 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -330> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -329> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -328> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -327> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): get_auth_request con 0x55a4cc7d7160 auth_method 0 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -326> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.233 INFO:tasks.workunit.client.0.smithi047.stderr: -325> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -324> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -323> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -322> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -321> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -320> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -319> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -318> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -317> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_done global_id 6140 payload 824 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -316> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -315> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 1 monclient: found mon.noname-c 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -314> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.234 INFO:tasks.workunit.client.0.smithi047.stderr: -313> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr: -312> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr: -311> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: dump: 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr: -310> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr: -309> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.235 INFO:tasks.workunit.client.0.smithi047.stderr: -308> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr: -307> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr: -306> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr: -305> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: dump: 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.236 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -304> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6140 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -303> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -302> 2024-03-04T18:04:47.117+0000 7fc7bb5e9640 4 set_mon_vals no callback set 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -301> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -300> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -299> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -298> 2024-03-04T18:04:47.121+0000 7fc69ffff640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -297> 2024-03-04T18:04:47.121+0000 7fc69ffff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -296> 2024-03-04T18:04:47.121+0000 7fc69ffff640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -295> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x7fc694065880 auth_method 0 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -294> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: get_monmap_and_config 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -293> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: build_initial_monmap 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -292> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 1 build_initial for_mkfs: 0 2024-03-04T18:04:47.237 INFO:tasks.workunit.client.0.smithi047.stderr: -291> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: monmap: 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr: -290> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr: -289> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr: -288> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.238 INFO:tasks.workunit.client.0.smithi047.stderr: -287> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -286> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -285> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -284> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -283> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -282> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -281> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -280> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -279> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -278> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -277> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -276> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -275> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -274> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: init 2024-03-04T18:04:47.239 INFO:tasks.workunit.client.0.smithi047.stderr: -273> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding auth protocol: cephx 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -272> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding auth protocol: cephx 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -271> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding auth protocol: cephx 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -270> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -269> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -268> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -267> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -266> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -265> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -264> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -263> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -262> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -261> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -260> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.240 INFO:tasks.workunit.client.0.smithi047.stderr: -259> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -258> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -257> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -256> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command rotate-key cmddesc rotate-key hook 0x7fff70d81bf8 EEXIST 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -255> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -254> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: _add_conns ranks=[2,0,1] 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -253> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-c con 0x55a4cc82dfd0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -252> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-a con 0x55a4cc7fbd90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -251> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-b con 0x55a4cc829a00 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -250> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -249> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -248> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -247> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -246> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): authenticate will time out at 2458.299268s 2024-03-04T18:04:47.241 INFO:tasks.workunit.client.0.smithi047.stderr: -245> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request con 0x55a4cc82dfd0 auth_method 0 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -244> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -243> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -242> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -241> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request con 0x55a4cc7fbd90 auth_method 0 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -240> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -239> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -238> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -237> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -236> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -235> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -234> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -233> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -232> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.242 INFO:tasks.workunit.client.0.smithi047.stderr: -231> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_done global_id 6143 payload 274 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr: -230> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr: -229> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 1 monclient: found mon.noname-c 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr: -228> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr: -227> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr: -226> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr: -225> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: dump: 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.243 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr: -224> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr: -223> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr: -222> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr: -221> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr: -220> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr: -219> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: dump: 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.244 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr: -218> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6143 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr: -217> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr: -216> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 4 set_mon_vals no callback set 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr: -215> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: shutdown 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr: -214> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: build_initial_monmap 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr: -213> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 build_initial for_mkfs: 0 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr: -212> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: monmap: 2024-03-04T18:04:47.245 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr: -211> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr: -210> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr: -209> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr: -208> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.246 INFO:tasks.workunit.client.0.smithi047.stderr: -207> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -206> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -205> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -204> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -203> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -202> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -201> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -200> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -199> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -198> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -197> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -196> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -195> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: starting msgr at 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -194> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: starting objecter 2024-03-04T18:04:47.247 INFO:tasks.workunit.client.0.smithi047.stderr: -193> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command objecter_requests cmddesc objecter_requests hook 0x55a4cc7db5a0 EEXIST 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -192> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: setting wanted keys 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -191> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: calling monclient init 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -190> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: init 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -189> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding auth protocol: cephx 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -188> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding auth protocol: cephx 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -187> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding auth protocol: cephx 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -186> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -185> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -184> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -183> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -182> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -181> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -180> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -179> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.248 INFO:tasks.workunit.client.0.smithi047.stderr: -178> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -177> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -176> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -175> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -174> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -173> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -172> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command rotate-key cmddesc rotate-key hook 0x55a4cc7f8a28 EEXIST 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -171> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -170> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _add_conns ranks=[2,1,0] 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -169> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-c con 0x55a4cc7fbd90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -168> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-b con 0x55a4cc829a00 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -167> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-a con 0x55a4cc82dfd0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -166> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.249 INFO:tasks.workunit.client.0.smithi047.stderr: -165> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -164> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -163> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -162> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): authenticate will time out at 2458.301928s 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -161> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request con 0x55a4cc7fbd90 auth_method 0 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -160> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -159> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -158> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -157> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): get_auth_request con 0x55a4cc82dfd0 auth_method 0 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -156> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -155> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -154> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -153> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.250 INFO:tasks.workunit.client.0.smithi047.stderr: -152> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -151> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -150> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -149> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -148> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -147> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_done global_id 6146 payload 824 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -146> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -145> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 1 monclient: found mon.noname-c 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -144> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -143> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -142> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr: -141> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: dump: 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.251 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr: -140> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr: -139> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr: -138> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr: -137> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr: -136> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr: -135> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: dump: 2024-03-04T18:04:47.252 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr: -134> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6146 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr: -133> 2024-03-04T18:04:47.125+0000 7fc69dffb640 4 set_mon_vals no callback set 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr: -132> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr: -131> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr: -130> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi047.stderr: -129> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -128> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: init done 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -127> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cr dump hook 0x55a4cc83c648 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -126> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -125> 2024-03-04T18:04:47.129+0000 7fc68bfff640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -124> 2024-03-04T18:04:47.129+0000 7fc68bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -123> 2024-03-04T18:04:47.129+0000 7fc68bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -122> 2024-03-04T18:04:47.129+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x7fc684065970 auth_method 0 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -121> 2024-03-04T18:04:47.129+0000 7fc7bade8640 10 monclient: get_auth_request con 0x55a4cc847ec0 auth_method 0 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -120> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -119> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 realm 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -118> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -117> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -116> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:47.254 INFO:tasks.workunit.client.0.smithi047.stderr: -115> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -114> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -113> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -112> 2024-03-04T18:04:47.133+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x55a4cc84c110 auth_method 0 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -111> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -110> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -109> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -108> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 searching for the correct realm 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -107> 2024-03-04T18:04:47.133+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x55a4cc851340 auth_method 0 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -106> 2024-03-04T18:04:47.133+0000 7fc7bade8640 10 monclient: get_auth_request con 0x7fc7b003a070 auth_method 0 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -105> 2024-03-04T18:04:47.137+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x7fc7ac0dc030 auth_method 0 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -104> 2024-03-04T18:04:47.137+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x7fc7b0083c20 auth_method 0 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -103> 2024-03-04T18:04:47.141+0000 7fc7bade8640 0 -- 172.21.15.47:0/2913834822 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55a4cc847ec0 msgr2=0x55a4cc84a310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:47.255 INFO:tasks.workunit.client.0.smithi047.stderr: -102> 2024-03-04T18:04:47.141+0000 7fc68bfff640 1 client.6146.objecter ms_handle_reset 0x55a4cc847ec0 session 0x55a4cc847840 osd.6 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -101> 2024-03-04T18:04:47.141+0000 7fc68bfff640 10 monclient: _renew_subs 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -100> 2024-03-04T18:04:47.141+0000 7fc68bfff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -99> 2024-03-04T18:04:47.141+0000 7fc7bade8640 10 monclient: get_auth_request con 0x7fc684075c90 auth_method 0 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -98> 2024-03-04T18:04:47.145+0000 7fc7bade8640 10 monclient: get_auth_request con 0x7fc7ac0c3040 auth_method 0 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -97> 2024-03-04T18:04:47.145+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x7fc7b4055640 auth_method 0 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -96> 2024-03-04T18:04:47.145+0000 7fc7ba5e7640 0 -- 172.21.15.47:0/2913834822 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x55a4cc84c110 msgr2=0x55a4cc84e540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -95> 2024-03-04T18:04:47.145+0000 7fc68bfff640 1 client.6146.objecter ms_handle_reset 0x55a4cc84c110 session 0x55a4cc84ba90 osd.1 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -94> 2024-03-04T18:04:47.149+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x55a4cc847ec0 auth_method 0 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -93> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -92> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -91> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -90> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:47.256 INFO:tasks.workunit.client.0.smithi047.stderr: -89> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -88> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -87> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -86> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -85> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -84> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -83> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -82> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -81> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 zone default found 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -80> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 4 Realm: () 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -79> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -78> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -77> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:47.257 INFO:tasks.workunit.client.0.smithi047.stderr: -76> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 zonegroup default 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -75> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -74> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -73> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -72> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -71> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -70> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -69> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 started sync module instance, tier type = 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -68> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -67> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=0 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -66> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=1 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -65> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=2 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -64> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=3 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -63> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=4 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -62> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=5 2024-03-04T18:04:47.258 INFO:tasks.workunit.client.0.smithi047.stderr: -61> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=6 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -60> 2024-03-04T18:04:47.177+0000 7fc7bcacf980 20 add_watcher() i=7 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -59> 2024-03-04T18:04:47.177+0000 7fc7bcacf980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -58> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding auth protocol: cephx 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -57> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding auth protocol: cephx 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -56> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding auth protocol: cephx 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -55> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -54> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -53> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -52> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -51> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -50> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -49> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -48> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.259 INFO:tasks.workunit.client.0.smithi047.stderr: -47> 2024-03-04T18:04:47.181+0000 7fc670fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -46> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -45> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -44> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -43> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -42> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -41> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -40> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -39> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cache list hook 0x55a4cc835f60 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -38> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cache inspect hook 0x55a4cc835f60 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -37> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cache erase hook 0x55a4cc835f60 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -36> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cache zap hook 0x55a4cc835f60 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -35> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 note: GC not initialized 2024-03-04T18:04:47.260 INFO:tasks.workunit.client.0.smithi047.stderr: -34> 2024-03-04T18:04:47.181+0000 7fc5eb7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -33> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command sync trace show hook 0x7fc7ac0047f0 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -32> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command sync trace history hook 0x7fc7ac0047f0 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -31> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command sync trace active hook 0x7fc7ac0047f0 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -30> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command sync trace active_short hook 0x7fc7ac0047f0 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -29> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -28> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 Filter name: none 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -27> 2024-03-04T18:04:47.181+0000 7fc5e97fa640 20 reqs_thread_entry: start 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -26> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -25> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -24> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -23> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -22> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -21> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 : miss 2024-03-04T18:04:47.261 INFO:tasks.workunit.client.0.smithi047.stderr: -20> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -19> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -18> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -17> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -16> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -15> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -14> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -13> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -12> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -11> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -10> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23, ofs0, ns] 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -9> 2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595112 cookie 94166294289264 notifier 6146 bl.length()=877 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -8> 2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -7> 2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi047.stderr: -6> 2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: -5> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_sources= new_sources= 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: -4> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential sources added=[] removed=[] 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: -3> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_dests= new_dests= 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: -2> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential dests added=[] removed=[] 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: -1> 2024-03-04T18:04:47.201+0000 7fc7bcacf980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fc7bcacf980 time 2024-03-04T18:04:47.203340+0000 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fc7c03a4d7b] 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fc7c03a4f2f] 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a4cb00c281] 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x55a4cb018ced] 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc7bf20fd90] 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 0> 2024-03-04T18:04:47.201+0000 7fc7bcacf980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7fc7bcacf980 thread_name:radosgw-admin 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fc7bf228520] 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fc7c03a4dd5] 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fc7c03a4f2f] 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a4cb00c281] 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x55a4cb018ced] 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc7bf20fd90] 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr:--- logging levels --- 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 none 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 lockdep 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 context 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 crush 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_balancer 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_locker 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log_expire 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_migrator 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 buffer 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 timer 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 filer 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 striper 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 objecter 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rados 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_mirror 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_replay 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_pwl 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 journaler 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objectcacher 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 client 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 osd 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 optracker 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objclass 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 filestore 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 journal 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 ms 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mon 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 0/10 monc 2024-03-04T18:04:47.267 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 paxos 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 tp 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 auth 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 crypto 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 finisher 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 reserver 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 heartbeatmap 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 perfcounter 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 20/20 rgw 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_sync 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_datacache 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_access 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:04:47.268 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_flight 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 javaclient 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 asok 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 throttle 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 refs 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 compressor 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluestore 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluefs 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 bdev 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 kstore 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 4/ 5 rocksdb 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 fuse 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 2/ 5 mgr 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mgrc 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 dpdk 2024-03-04T18:04:47.269 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 eventtrace 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 prioritycache 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 test 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephsqlite 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_onode 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_odata 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_omap 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_tm 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_t 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_epm 2024-03-04T18:04:47.270 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_lba 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cache 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_journal 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_device 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_backref 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 alienstore 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mclock 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cyanstore 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 ceph_exporter 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 memstore 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 trace 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: -2/-2 (syslog threshold) 2024-03-04T18:04:47.271 INFO:tasks.workunit.client.0.smithi047.stderr: 99/99 (stderr threshold) 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc5e97fa640 / http_manager 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc5eb7fe640 / http_manager 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc670fe1640 / rgw_dt_lg_renew 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc68bfff640 / ms_dispatch 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc69dffb640 / io_context_pool 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc69ffff640 / ms_dispatch 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc6a6e05640 / ms_dispatch 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc6a8608640 / io_context_pool 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7b9de6640 / msgr-worker-2 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7ba5e7640 / msgr-worker-1 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7bade8640 / msgr-worker-0 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7bb5e9640 / io_context_pool 2024-03-04T18:04:47.272 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7bcacf980 / radosgw-admin 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: max_recent 500 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: max_new 1000 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: log_file /var/log/ceph/ceph-client.admin.85809.log 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr:--- end dump of recent events --- 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr:--- begin dump of recent events --- 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: -499> 2024-03-04T18:04:47.105+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: -498> 2024-03-04T18:04:47.105+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: -497> 2024-03-04T18:04:47.105+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_done global_id 6510 payload 274 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: -496> 2024-03-04T18:04:47.105+0000 7fc7b9de6640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: -495> 2024-03-04T18:04:47.105+0000 7fc7b9de6640 1 monclient: found mon.noname-a 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: -494> 2024-03-04T18:04:47.105+0000 7fc7b9de6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: -493> 2024-03-04T18:04:47.105+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x55a4cc61b240 auth_method 0 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: -492> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.273 INFO:tasks.workunit.client.0.smithi047.stderr: -491> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr: -490> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: dump: 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr: -489> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr: -488> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.274 INFO:tasks.workunit.client.0.smithi047.stderr: -487> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr: -486> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr: -485> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr: -484> 2024-03-04T18:04:47.105+0000 7fc6a8608640 10 monclient: dump: 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.275 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr: -483> 2024-03-04T18:04:47.105+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6510 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr: -482> 2024-03-04T18:04:47.105+0000 7fc7bcacf980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr: -481> 2024-03-04T18:04:47.105+0000 7fc7bcacf980 4 set_mon_vals no callback set 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr: -480> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: shutdown 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr: -479> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 asok(0x55a4cc578130) unregister_commands rotate-key 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr: -478> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: build_initial_monmap 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr: -477> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 build_initial for_mkfs: 0 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr: -476> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: monmap: 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.276 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -475> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding auth protocol: cephx 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -474> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding auth protocol: cephx 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -473> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding auth protocol: cephx 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -472> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -471> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -470> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -469> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -468> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -467> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.277 INFO:tasks.workunit.client.0.smithi047.stderr: -466> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -465> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -464> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -463> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -462> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: crc 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -461> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc763340) adding con mode: secure 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -460> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -459> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: starting msgr at 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -458> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: starting objecter 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -457> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command objecter_requests hook 0x55a4cc522260 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -456> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: setting wanted keys 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -455> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: calling monclient init 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -454> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: init 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -453> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding auth protocol: cephx 2024-03-04T18:04:47.278 INFO:tasks.workunit.client.0.smithi047.stderr: -452> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding auth protocol: cephx 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -451> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding auth protocol: cephx 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -450> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -449> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -448> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -447> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -446> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -445> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -444> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -443> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -442> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -441> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -440> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: crc 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -439> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc75d600) adding con mode: secure 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -438> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.279 INFO:tasks.workunit.client.0.smithi047.stderr: -437> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -436> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command rotate-key hook 0x55a4cc75d748 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -435> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -434> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _add_conns ranks=[2,1,0] 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -433> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-c con 0x55a4cc61b240 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -432> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-b con 0x55a4cc760630 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -431> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-a con 0x55a4cc78d4b0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -430> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -429> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -428> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -427> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -426> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient(hunting): authenticate will time out at 2458.285775s 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -425> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): get_auth_request con 0x55a4cc78d4b0 auth_method 0 2024-03-04T18:04:47.280 INFO:tasks.workunit.client.0.smithi047.stderr: -424> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -423> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -422> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -421> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request con 0x55a4cc61b240 auth_method 0 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -420> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -419> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -418> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -417> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -416> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -415> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -414> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -413> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -412> 2024-03-04T18:04:47.109+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.281 INFO:tasks.workunit.client.0.smithi047.stderr: -411> 2024-03-04T18:04:47.109+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request con 0x55a4cc760630 auth_method 0 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -410> 2024-03-04T18:04:47.109+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -409> 2024-03-04T18:04:47.109+0000 7fc7b9de6640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -408> 2024-03-04T18:04:47.109+0000 7fc7b9de6640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -407> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient(hunting): handle_auth_done global_id 6513 payload 824 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -406> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -405> 2024-03-04T18:04:47.109+0000 7fc7bade8640 1 monclient: found mon.noname-a 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -404> 2024-03-04T18:04:47.109+0000 7fc7bade8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -403> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -402> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr: -401> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: dump: 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.282 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr: -400> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr: -399> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr: -398> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr: -397> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr: -396> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr: -395> 2024-03-04T18:04:47.109+0000 7fc6a6e05640 10 monclient: dump: 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.283 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr: -394> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6513 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr: -393> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr: -392> 2024-03-04T18:04:47.109+0000 7fc6a8608640 4 set_mon_vals no callback set 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr: -391> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:47.284 INFO:tasks.workunit.client.0.smithi047.stderr: -390> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -389> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -388> 2024-03-04T18:04:47.109+0000 7fc7bcacf980 1 librados: init done 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -387> 2024-03-04T18:04:47.113+0000 7fc6a6e05640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -386> 2024-03-04T18:04:47.113+0000 7fc6a6e05640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -385> 2024-03-04T18:04:47.113+0000 7fc6a6e05640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -384> 2024-03-04T18:04:47.113+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x7fc6a0065970 auth_method 0 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -383> 2024-03-04T18:04:47.113+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x55a4cc79bd80 auth_method 0 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -382> 2024-03-04T18:04:47.113+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x55a4cc7a8ab0 auth_method 0 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -381> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x55a4cc7adb40 auth_method 0 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -380> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: build_initial_monmap 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -379> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 1 build_initial for_mkfs: 0 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr: -378> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: monmap: 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:47.285 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr: -377> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding auth protocol: cephx 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr: -376> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding auth protocol: cephx 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr: -375> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding auth protocol: cephx 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr: -374> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr: -373> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr: -372> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr: -371> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.286 INFO:tasks.workunit.client.0.smithi047.stderr: -370> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -369> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -368> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -367> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -366> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -365> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -364> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: crc 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -363> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b17b0) adding con mode: secure 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -362> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -361> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command objecter_requests cmddesc objecter_requests hook 0x55a4cc7b6f60 EEXIST 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -360> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: init 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -359> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding auth protocol: cephx 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -358> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding auth protocol: cephx 2024-03-04T18:04:47.287 INFO:tasks.workunit.client.0.smithi047.stderr: -357> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding auth protocol: cephx 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -356> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -355> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -354> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -353> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -352> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -351> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -350> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -349> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -348> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -347> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -346> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: crc 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -345> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7b0888) adding con mode: secure 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -344> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi047.stderr: -343> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -342> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command rotate-key cmddesc rotate-key hook 0x55a4cc7b09d0 EEXIST 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -341> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -340> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -339> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-b con 0x55a4cc7dbbc0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -338> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-c con 0x55a4cc7dbfa0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -337> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-a con 0x55a4cc7d7160 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -336> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -335> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -334> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -333> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -332> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient(hunting): authenticate will time out at 2458.295029s 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -331> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request con 0x55a4cc7dbfa0 auth_method 0 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -330> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi047.stderr: -329> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -328> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -327> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): get_auth_request con 0x55a4cc7d7160 auth_method 0 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -326> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -325> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -324> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -323> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -322> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -321> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -320> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -319> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -318> 2024-03-04T18:04:47.117+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -317> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_done global_id 6140 payload 824 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -316> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -315> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 1 monclient: found mon.noname-c 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi047.stderr: -314> 2024-03-04T18:04:47.117+0000 7fc7b9de6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr: -313> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr: -312> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr: -311> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: dump: 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.291 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr: -310> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr: -309> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr: -308> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr: -307> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr: -306> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr: -305> 2024-03-04T18:04:47.117+0000 7fc69ffff640 10 monclient: dump: 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.292 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -304> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6140 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -303> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -302> 2024-03-04T18:04:47.117+0000 7fc7bb5e9640 4 set_mon_vals no callback set 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -301> 2024-03-04T18:04:47.117+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -300> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -299> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -298> 2024-03-04T18:04:47.121+0000 7fc69ffff640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -297> 2024-03-04T18:04:47.121+0000 7fc69ffff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -296> 2024-03-04T18:04:47.121+0000 7fc69ffff640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -295> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x7fc694065880 auth_method 0 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -294> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: get_monmap_and_config 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -293> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: build_initial_monmap 2024-03-04T18:04:47.293 INFO:tasks.workunit.client.0.smithi047.stderr: -292> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 1 build_initial for_mkfs: 0 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr: -291> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: monmap: 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr: -290> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.294 INFO:tasks.workunit.client.0.smithi047.stderr: -289> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -288> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -287> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -286> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -285> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -284> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -283> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -282> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -281> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -280> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -279> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -278> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -277> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -276> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi047.stderr: -275> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -274> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: init 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -273> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding auth protocol: cephx 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -272> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding auth protocol: cephx 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -271> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding auth protocol: cephx 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -270> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -269> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -268> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -267> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -266> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -265> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -264> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -263> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi047.stderr: -262> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -261> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -260> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: crc 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -259> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81ab0) adding con mode: secure 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -258> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -257> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -256> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command rotate-key cmddesc rotate-key hook 0x7fff70d81bf8 EEXIST 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -255> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -254> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient: _add_conns ranks=[2,0,1] 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -253> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-c con 0x55a4cc82dfd0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -252> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-a con 0x55a4cc7fbd90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -251> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-b con 0x55a4cc829a00 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -250> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -249> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi047.stderr: -248> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -247> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -246> 2024-03-04T18:04:47.121+0000 7fc7bcacf980 10 monclient(hunting): authenticate will time out at 2458.299268s 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -245> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request con 0x55a4cc82dfd0 auth_method 0 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -244> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -243> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -242> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -241> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request con 0x55a4cc7fbd90 auth_method 0 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -240> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -239> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -238> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -237> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -236> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -235> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi047.stderr: -234> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr: -233> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr: -232> 2024-03-04T18:04:47.121+0000 7fc7b9de6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr: -231> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_done global_id 6143 payload 274 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr: -230> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr: -229> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 1 monclient: found mon.noname-c 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr: -228> 2024-03-04T18:04:47.121+0000 7fc7ba5e7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr: -227> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr: -226> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr: -225> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: dump: 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr: -224> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr: -223> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr: -222> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr: -221> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr: -220> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr: -219> 2024-03-04T18:04:47.125+0000 7fc69dffb640 10 monclient: dump: 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.300 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr: -218> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6143 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr: -217> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr: -216> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 4 set_mon_vals no callback set 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr: -215> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: shutdown 2024-03-04T18:04:47.301 INFO:tasks.workunit.client.0.smithi047.stderr: -214> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: build_initial_monmap 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr: -213> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 build_initial for_mkfs: 0 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr: -212> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: monmap: 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr: -211> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr: -210> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr: -209> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding auth protocol: cephx 2024-03-04T18:04:47.302 INFO:tasks.workunit.client.0.smithi047.stderr: -208> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -207> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -206> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -205> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -204> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -203> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -202> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -201> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -200> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -199> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -198> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: crc 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -197> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f9710) adding con mode: secure 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -196> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -195> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: starting msgr at 2024-03-04T18:04:47.303 INFO:tasks.workunit.client.0.smithi047.stderr: -194> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: starting objecter 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -193> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command objecter_requests cmddesc objecter_requests hook 0x55a4cc7db5a0 EEXIST 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -192> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: setting wanted keys 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -191> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: calling monclient init 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -190> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: init 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -189> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding auth protocol: cephx 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -188> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding auth protocol: cephx 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -187> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding auth protocol: cephx 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -186> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -185> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -184> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -183> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -182> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -181> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.304 INFO:tasks.workunit.client.0.smithi047.stderr: -180> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -179> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -178> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -177> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -176> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: crc 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -175> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 AuthRegistry(0x55a4cc7f88e0) adding con mode: secure 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -174> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -173> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -172> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command rotate-key cmddesc rotate-key hook 0x55a4cc7f8a28 EEXIST 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -171> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -170> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _add_conns ranks=[2,1,0] 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -169> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-c con 0x55a4cc7fbd90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -168> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-b con 0x55a4cc829a00 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -167> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): picked mon.noname-a con 0x55a4cc82dfd0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi047.stderr: -166> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -165> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -164> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -163> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -162> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient(hunting): authenticate will time out at 2458.301928s 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -161> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request con 0x55a4cc7fbd90 auth_method 0 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -160> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -159> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -158> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -157> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): get_auth_request con 0x55a4cc82dfd0 auth_method 0 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -156> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -155> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -154> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -153> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi047.stderr: -152> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -151> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -150> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -149> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -148> 2024-03-04T18:04:47.125+0000 7fc7bade8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -147> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient(hunting): handle_auth_done global_id 6146 payload 824 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -146> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient: _finish_hunting 0 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -145> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 1 monclient: found mon.noname-c 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -144> 2024-03-04T18:04:47.125+0000 7fc7ba5e7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -143> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -142> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr: -141> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: dump: 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.307 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr: -140> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: _finish_auth 0 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr: -139> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: _check_auth_tickets 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr: -138> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr: -137> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr: -136> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:47.308 INFO:tasks.workunit.client.0.smithi047.stderr: -135> 2024-03-04T18:04:47.125+0000 7fc68bfff640 10 monclient: dump: 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr: -134> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 monclient: authenticate success, global_id 6146 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr: -133> 2024-03-04T18:04:47.125+0000 7fc69dffb640 4 set_mon_vals no callback set 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr: -132> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr: -131> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi047.stderr: -130> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _renew_subs 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -129> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -128> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 1 librados: init done 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -127> 2024-03-04T18:04:47.125+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cr dump hook 0x55a4cc83c648 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -126> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -125> 2024-03-04T18:04:47.129+0000 7fc68bfff640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -124> 2024-03-04T18:04:47.129+0000 7fc68bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -123> 2024-03-04T18:04:47.129+0000 7fc68bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -122> 2024-03-04T18:04:47.129+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x7fc684065970 auth_method 0 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -121> 2024-03-04T18:04:47.129+0000 7fc7bade8640 10 monclient: get_auth_request con 0x55a4cc847ec0 auth_method 0 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -120> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -119> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 realm 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -118> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -117> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.310 INFO:tasks.workunit.client.0.smithi047.stderr: -116> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -115> 2024-03-04T18:04:47.129+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -114> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -113> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -112> 2024-03-04T18:04:47.133+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x55a4cc84c110 auth_method 0 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -111> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -110> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -109> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -108> 2024-03-04T18:04:47.133+0000 7fc7bcacf980 20 searching for the correct realm 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -107> 2024-03-04T18:04:47.133+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x55a4cc851340 auth_method 0 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -106> 2024-03-04T18:04:47.133+0000 7fc7bade8640 10 monclient: get_auth_request con 0x7fc7b003a070 auth_method 0 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -105> 2024-03-04T18:04:47.137+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x7fc7ac0dc030 auth_method 0 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -104> 2024-03-04T18:04:47.137+0000 7fc7b9de6640 10 monclient: get_auth_request con 0x7fc7b0083c20 auth_method 0 2024-03-04T18:04:47.311 INFO:tasks.workunit.client.0.smithi047.stderr: -103> 2024-03-04T18:04:47.141+0000 7fc7bade8640 0 -- 172.21.15.47:0/2913834822 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55a4cc847ec0 msgr2=0x55a4cc84a310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -102> 2024-03-04T18:04:47.141+0000 7fc68bfff640 1 client.6146.objecter ms_handle_reset 0x55a4cc847ec0 session 0x55a4cc847840 osd.6 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -101> 2024-03-04T18:04:47.141+0000 7fc68bfff640 10 monclient: _renew_subs 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -100> 2024-03-04T18:04:47.141+0000 7fc68bfff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -99> 2024-03-04T18:04:47.141+0000 7fc7bade8640 10 monclient: get_auth_request con 0x7fc684075c90 auth_method 0 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -98> 2024-03-04T18:04:47.145+0000 7fc7bade8640 10 monclient: get_auth_request con 0x7fc7ac0c3040 auth_method 0 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -97> 2024-03-04T18:04:47.145+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x7fc7b4055640 auth_method 0 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -96> 2024-03-04T18:04:47.145+0000 7fc7ba5e7640 0 -- 172.21.15.47:0/2913834822 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x55a4cc84c110 msgr2=0x55a4cc84e540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -95> 2024-03-04T18:04:47.145+0000 7fc68bfff640 1 client.6146.objecter ms_handle_reset 0x55a4cc84c110 session 0x55a4cc84ba90 osd.1 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -94> 2024-03-04T18:04:47.149+0000 7fc7ba5e7640 10 monclient: get_auth_request con 0x55a4cc847ec0 auth_method 0 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -93> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -92> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -91> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi047.stderr: -90> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -89> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -88> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -87> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -86> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -85> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -84> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -83> 2024-03-04T18:04:47.153+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -82> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -81> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 zone default found 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -80> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 4 Realm: () 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -79> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -78> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -77> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -76> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 zonegroup default 2024-03-04T18:04:47.313 INFO:tasks.workunit.client.0.smithi047.stderr: -75> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -74> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -73> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -72> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -71> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -70> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -69> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 started sync module instance, tier type = 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -68> 2024-03-04T18:04:47.157+0000 7fc7bcacf980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -67> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=0 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -66> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=1 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -65> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=2 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -64> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=3 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -63> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=4 2024-03-04T18:04:47.314 INFO:tasks.workunit.client.0.smithi047.stderr: -62> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=5 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -61> 2024-03-04T18:04:47.173+0000 7fc7bcacf980 20 add_watcher() i=6 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -60> 2024-03-04T18:04:47.177+0000 7fc7bcacf980 20 add_watcher() i=7 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -59> 2024-03-04T18:04:47.177+0000 7fc7bcacf980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -58> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding auth protocol: cephx 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -57> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding auth protocol: cephx 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -56> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding auth protocol: cephx 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -55> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -54> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -53> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -52> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -51> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -50> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -49> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -48> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -47> 2024-03-04T18:04:47.181+0000 7fc670fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.315 INFO:tasks.workunit.client.0.smithi047.stderr: -46> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -45> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -44> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: crc 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -43> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 AuthRegistry(0x7fff70d81f40) adding con mode: secure 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -42> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -41> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -40> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -39> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cache list hook 0x55a4cc835f60 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -38> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cache inspect hook 0x55a4cc835f60 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -37> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cache erase hook 0x55a4cc835f60 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -36> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command cache zap hook 0x55a4cc835f60 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -35> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 note: GC not initialized 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -34> 2024-03-04T18:04:47.181+0000 7fc5eb7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:47.316 INFO:tasks.workunit.client.0.smithi047.stderr: -33> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command sync trace show hook 0x7fc7ac0047f0 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -32> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command sync trace history hook 0x7fc7ac0047f0 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -31> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command sync trace active hook 0x7fc7ac0047f0 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -30> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 5 asok(0x55a4cc578130) register_command sync trace active_short hook 0x7fc7ac0047f0 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -29> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -28> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 Filter name: none 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -27> 2024-03-04T18:04:47.181+0000 7fc5e97fa640 20 reqs_thread_entry: start 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -26> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -25> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -24> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -23> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -22> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -21> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 : miss 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi047.stderr: -20> 2024-03-04T18:04:47.181+0000 7fc7bcacf980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -19> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -18> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -17> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -16> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -15> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -14> 2024-03-04T18:04:47.185+0000 7fc7bcacf980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -13> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -12> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -11> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -10> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23, ofs0, ns] 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -9> 2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595112 cookie 94166294289264 notifier 6146 bl.length()=877 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -8> 2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -7> 2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.318 INFO:tasks.workunit.client.0.smithi047.stderr: -6> 2024-03-04T18:04:47.197+0000 7fc69dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: -5> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_sources= new_sources= 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: -4> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential sources added=[] removed=[] 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: -3> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_dests= new_dests= 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: -2> 2024-03-04T18:04:47.197+0000 7fc7bcacf980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential dests added=[] removed=[] 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: -1> 2024-03-04T18:04:47.201+0000 7fc7bcacf980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fc7bcacf980 time 2024-03-04T18:04:47.203340+0000 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fc7c03a4d7b] 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fc7c03a4f2f] 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a4cb00c281] 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x55a4cb018ced] 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:04:47.319 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc7bf20fd90] 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 0> 2024-03-04T18:04:47.201+0000 7fc7bcacf980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7fc7bcacf980 thread_name:radosgw-admin 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fc7bf228520] 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fc7c03a4dd5] 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fc7c03a4f2f] 2024-03-04T18:04:47.320 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a4cb00c281] 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x55a4cb018ced] 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc7bf20fd90] 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr:--- logging levels --- 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 none 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 lockdep 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 context 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 crush 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds 2024-03-04T18:04:47.321 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_balancer 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_locker 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log_expire 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_migrator 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 buffer 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 timer 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 filer 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 striper 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 objecter 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rados 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_mirror 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_replay 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_pwl 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 journaler 2024-03-04T18:04:47.322 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objectcacher 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 client 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 osd 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 optracker 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objclass 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 filestore 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 journal 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 ms 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mon 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 0/10 monc 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 paxos 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 tp 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 auth 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 crypto 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 finisher 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 reserver 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 heartbeatmap 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 perfcounter 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 20/20 rgw 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_sync 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_datacache 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_access 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_flight 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 javaclient 2024-03-04T18:04:47.324 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 asok 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 throttle 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 refs 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 compressor 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluestore 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluefs 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 bdev 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 kstore 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 4/ 5 rocksdb 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 fuse 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 2/ 5 mgr 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mgrc 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 dpdk 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 eventtrace 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 prioritycache 2024-03-04T18:04:47.325 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 test 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephsqlite 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_onode 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_odata 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_omap 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_tm 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_t 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_epm 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_lba 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:04:47.326 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cache 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_journal 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_device 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_backref 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 alienstore 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mclock 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cyanstore 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 ceph_exporter 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 memstore 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 trace 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: -2/-2 (syslog threshold) 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 99/99 (stderr threshold) 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc5e97fa640 / http_manager 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc5eb7fe640 / http_manager 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc670fe1640 / rgw_dt_lg_renew 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc68bfff640 / ms_dispatch 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc69dffb640 / io_context_pool 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc69ffff640 / ms_dispatch 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc6a6e05640 / ms_dispatch 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc6a8608640 / io_context_pool 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7b9de6640 / msgr-worker-2 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7ba5e7640 / msgr-worker-1 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7bade8640 / msgr-worker-0 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7bb5e9640 / io_context_pool 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: 7fc7bcacf980 / radosgw-admin 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: max_recent 500 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: max_new 1000 2024-03-04T18:04:47.328 INFO:tasks.workunit.client.0.smithi047.stderr: log_file /var/lib/ceph/crash/2024-03-04T18:04:47.205409Z_85ae2a84-866a-4930-9f0f-65cfc3fb9d2c/log 2024-03-04T18:04:47.329 INFO:tasks.workunit.client.0.smithi047.stderr:--- end dump of recent events --- 2024-03-04T18:04:47.329 INFO:tasks.workunit.client.0.smithi047.stderr:Aborted 2024-03-04T18:04:47.329 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-03-04T18:04:47.331 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.285+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 realm 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:47.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.289+0000 7fa64b215980 20 searching for the correct realm 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.293+0000 7fa648d2d640 0 -- 172.21.15.47:0/4284583646 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fa6380d2040 msgr2=0x7fa6380bf0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.301+0000 7fa64952e640 0 -- 172.21.15.47:0/4284583646 >> [v2:172.21.15.152:6818/159740591,v1:172.21.15.152:6821/159740591] conn(0x7fa638118070 msgr2=0x7fa638107040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.309+0000 7fa64b215980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.309+0000 7fa64b215980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.309+0000 7fa64b215980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.309+0000 7fa64b215980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.309+0000 7fa64b215980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.309+0000 7fa64b215980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.309+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.333 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 zone default found 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 4 Realm: () 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 zonegroup default 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 started sync module instance, tier type = 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.313+0000 7fa64b215980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.333+0000 7fa64b215980 20 add_watcher() i=0 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.333+0000 7fa64b215980 20 add_watcher() i=1 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.333+0000 7fa64b215980 20 add_watcher() i=2 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.333+0000 7fa64b215980 20 add_watcher() i=3 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.333+0000 7fa64b215980 20 add_watcher() i=4 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.333+0000 7fa64b215980 20 add_watcher() i=5 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.333+0000 7fa64b215980 20 add_watcher() i=6 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.333+0000 7fa64b215980 20 add_watcher() i=7 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.333+0000 7fa64b215980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:47.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.337+0000 7fa4fefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.337+0000 7fa64b215980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.337+0000 7fa64b215980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.337+0000 7fa64b215980 5 note: GC not initialized 2024-03-04T18:04:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.337+0000 7fa4fdfe3640 20 reqs_thread_entry: start 2024-03-04T18:04:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.337+0000 7fa64b215980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.337+0000 7fa64b215980 20 Filter name: none 2024-03-04T18:04:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.337+0000 7fa46f7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.337+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.341+0000 7fa64b215980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:47.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.341+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.341+0000 7fa64b215980 20 rados_obj.operate() r=0 bl.length=405 2024-03-04T18:04:47.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.341+0000 7fa64b215980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:47.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.341+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.341+0000 7fa64b215980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:47.339 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.341+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.345+0000 7fa64b215980 20 rados_obj.operate() r=0 bl.length=405 2024-03-04T18:04:47.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.345+0000 7fa64b215980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:47.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.345+0000 7fa64b215980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.342 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.345+0000 7fa64b215980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.349+0000 7fa4fefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.353+0000 7fa64b215980 20 remove_watcher() i=0 2024-03-04T18:04:47.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.353+0000 7fa64b215980 2 removed watcher, disabling cache 2024-03-04T18:04:47.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.353+0000 7fa64b215980 20 remove_watcher() i=1 2024-03-04T18:04:47.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.357+0000 7fa64b215980 20 remove_watcher() i=2 2024-03-04T18:04:47.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.357+0000 7fa64b215980 20 remove_watcher() i=3 2024-03-04T18:04:47.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.361+0000 7fa64b215980 20 remove_watcher() i=4 2024-03-04T18:04:47.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.361+0000 7fa64b215980 20 remove_watcher() i=5 2024-03-04T18:04:47.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.365+0000 7fa64b215980 20 remove_watcher() i=6 2024-03-04T18:04:47.364 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.369+0000 7fa64b215980 20 remove_watcher() i=7 2024-03-04T18:04:47.379 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.379 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:04:47.379 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-block-writes id: 797b67a4-b876-4d04-b122-10afd858d465.4453.23 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:04:47.379 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:04:47.380 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:47.380 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:47.413 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:47.413 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:04:47.469 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:47.505 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:47.505 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:47.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.553+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.553+0000 7f766d0a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.553+0000 7f766d0a8980 20 realm 2024-03-04T18:04:47.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.553+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.553+0000 7f766d0a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.553+0000 7f766d0a8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:47.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.553+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.557+0000 7f766d0a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.557+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.557+0000 7f766d0a8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.557+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.557+0000 7f766d0a8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:47.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.557+0000 7f766d0a8980 20 searching for the correct realm 2024-03-04T18:04:47.560 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.565+0000 7f766a3bf640 0 -- 172.21.15.47:0/2993932130 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f765c01c3c0 msgr2=0x7f765c01c840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.569+0000 7f766b3c1640 0 -- 172.21.15.47:0/2993932130 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55795a14caf0 msgr2=0x55795a14efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:04:47.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:04:47.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:47.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:47.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:47.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:47.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:47.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 zone default found 2024-03-04T18:04:47.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 4 Realm: () 2024-03-04T18:04:47.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:04:47.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:04:47.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:47.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 zonegroup default 2024-03-04T18:04:47.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 started sync module instance, tier type = 2024-03-04T18:04:47.579 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.581+0000 7f766d0a8980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:04:47.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.601+0000 7f766d0a8980 20 add_watcher() i=0 2024-03-04T18:04:47.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.601+0000 7f766d0a8980 20 add_watcher() i=1 2024-03-04T18:04:47.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.601+0000 7f766d0a8980 20 add_watcher() i=2 2024-03-04T18:04:47.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.601+0000 7f766d0a8980 20 add_watcher() i=3 2024-03-04T18:04:47.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.601+0000 7f766d0a8980 20 add_watcher() i=4 2024-03-04T18:04:47.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.601+0000 7f766d0a8980 20 add_watcher() i=5 2024-03-04T18:04:47.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.601+0000 7f766d0a8980 20 add_watcher() i=6 2024-03-04T18:04:47.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.605+0000 7f766d0a8980 20 add_watcher() i=7 2024-03-04T18:04:47.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.605+0000 7f766d0a8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:47.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f75217e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f766d0a8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:47.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f766d0a8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:47.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f766d0a8980 5 note: GC not initialized 2024-03-04T18:04:47.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f749bfff640 20 reqs_thread_entry: start 2024-03-04T18:04:47.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f766d0a8980 20 init_complete bucket index max shards: 11 2024-03-04T18:04:47.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f766d0a8980 20 Filter name: none 2024-03-04T18:04:47.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f7499ffb640 20 reqs_thread_entry: start 2024-03-04T18:04:47.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f766d0a8980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:04:47.607 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.609+0000 7f766d0a8980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:04:47.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:04:47.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:04:47.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 : miss 2024-03-04T18:04:47.608 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 20 rados_obj.operate() r=0 bl.length=405 2024-03-04T18:04:47.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:04:47.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:04:47.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:47.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:04:47.609 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.613+0000 7f766d0a8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 2024-03-04T18:04:47.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.617+0000 7f766d0a8980 0 INFO: RGWReshardLock::lock found lock on abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to be held by another RGW process; skipping for now 2024-03-04T18:04:47.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.617+0000 7f75217e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.621+0000 7f766d0a8980 20 remove_watcher() i=0 2024-03-04T18:04:47.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.621+0000 7f766d0a8980 2 removed watcher, disabling cache 2024-03-04T18:04:47.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.625+0000 7f766d0a8980 20 remove_watcher() i=1 2024-03-04T18:04:47.623 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.625+0000 7f766d0a8980 20 remove_watcher() i=2 2024-03-04T18:04:47.626 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.629+0000 7f766d0a8980 20 remove_watcher() i=3 2024-03-04T18:04:47.628 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.633+0000 7f766d0a8980 20 remove_watcher() i=4 2024-03-04T18:04:47.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.633+0000 7f766d0a8980 20 remove_watcher() i=5 2024-03-04T18:04:47.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.637+0000 7f766d0a8980 20 remove_watcher() i=6 2024-03-04T18:04:47.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:04:47.641+0000 7f766d0a8980 20 remove_watcher() i=7 2024-03-04T18:04:47.651 INFO:tasks.workunit.client.0.smithi047.stderr:waiting 30 seconds for reshard lock to expire... 2024-03-04T18:05:17.664 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:05:17.703 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:17.703 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:17.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.748+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.752+0000 7f03a6b86980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.752+0000 7f03a6b86980 20 realm 2024-03-04T18:05:17.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.752+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.752+0000 7f03a6b86980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.752+0000 7f03a6b86980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:17.750 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.752+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.752+0000 7f03a6b86980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.752+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.756+0000 7f03a6b86980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:17.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.756+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.756+0000 7f03a6b86980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:17.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.756+0000 7f03a6b86980 20 searching for the correct realm 2024-03-04T18:05:17.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.764+0000 7f03a4e9f640 0 -- 172.21.15.47:0/2044707842 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x561af1c3efd0 msgr2=0x561af1c41430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:17.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.772+0000 7f03a3e9d640 0 -- 172.21.15.47:0/2044707842 >> [v2:172.21.15.152:6808/929675582,v1:172.21.15.152:6809/929675582] conn(0x561af1c483f0 msgr2=0x561af1c4a880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:17.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:17.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:17.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:17.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:17.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:17.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:17.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:17.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.776+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:17.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 zone default found 2024-03-04T18:05:17.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 4 Realm: () 2024-03-04T18:05:17.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:17.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:17.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:17.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 zonegroup default 2024-03-04T18:05:17.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 started sync module instance, tier type = 2024-03-04T18:05:17.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.780+0000 7f03a6b86980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:17.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.800+0000 7f03a6b86980 20 add_watcher() i=0 2024-03-04T18:05:17.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.800+0000 7f03a6b86980 20 add_watcher() i=1 2024-03-04T18:05:17.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.800+0000 7f03a6b86980 20 add_watcher() i=2 2024-03-04T18:05:17.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.800+0000 7f03a6b86980 20 add_watcher() i=3 2024-03-04T18:05:17.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.800+0000 7f03a6b86980 20 add_watcher() i=4 2024-03-04T18:05:17.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.800+0000 7f03a6b86980 20 add_watcher() i=5 2024-03-04T18:05:17.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.800+0000 7f03a6b86980 20 add_watcher() i=6 2024-03-04T18:05:17.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.800+0000 7f03a6b86980 20 add_watcher() i=7 2024-03-04T18:05:17.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.800+0000 7f03a6b86980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:17.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f025afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:17.803 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f03a6b86980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:17.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f03a6b86980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:17.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f03a6b86980 5 note: GC not initialized 2024-03-04T18:05:17.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f0259fdb640 20 reqs_thread_entry: start 2024-03-04T18:05:17.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f03a6b86980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:17.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f03a6b86980 20 Filter name: none 2024-03-04T18:05:17.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f01cb7fe640 20 reqs_thread_entry: start 2024-03-04T18:05:17.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f03a6b86980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:05:17.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.804+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:05:17.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:05:17.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:05:17.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 : miss 2024-03-04T18:05:17.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 20 rados_obj.operate() r=0 bl.length=405 2024-03-04T18:05:17.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:05:17.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:05:17.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:17.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:17.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.808+0000 7f03a6b86980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 2024-03-04T18:05:17.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.812+0000 7f03a6b86980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-03-04T18:05:17.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f03a6b86980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:05:17.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f03a6b86980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:05:17.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f03a6b86980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:17.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f03a6b86980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23, ofs0, ns] 2024-03-04T18:05:17.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f027affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595113 cookie 94673725345920 notifier 6194 bl.length()=877 2024-03-04T18:05:17.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f027affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:05:17.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f027affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:05:17.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f027affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:17.827 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f03a6b86980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_sources= new_sources= 2024-03-04T18:05:17.827 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f03a6b86980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential sources added=[] removed=[] 2024-03-04T18:05:17.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f03a6b86980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_dests= new_dests= 2024-03-04T18:05:17.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.828+0000 7f03a6b86980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential dests added=[] removed=[] 2024-03-04T18:05:17.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.0 2024-03-04T18:05:17.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.0 2024-03-04T18:05:17.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.1 2024-03-04T18:05:17.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.1 2024-03-04T18:05:17.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.2 2024-03-04T18:05:17.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.2 2024-03-04T18:05:17.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.3 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.3 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.4 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.4 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.5 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.5 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.6 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.6 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.7 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.7 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.8 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.8 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.9 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.9 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.10 2024-03-04T18:05:17.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.10 2024-03-04T18:05:17.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.11 2024-03-04T18:05:17.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2.11 2024-03-04T18:05:17.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.0 2024-03-04T18:05:17.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.1 2024-03-04T18:05:17.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.836+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.2 2024-03-04T18:05:17.836 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.840+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.3 2024-03-04T18:05:17.837 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.840+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.4 2024-03-04T18:05:17.837 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.840+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.5 2024-03-04T18:05:17.839 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.840+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.6 2024-03-04T18:05:17.840 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.844+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.7 2024-03-04T18:05:17.841 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.844+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.8 2024-03-04T18:05:17.842 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.844+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.9 2024-03-04T18:05:17.843 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.844+0000 7f03a6b86980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.23.10 2024-03-04T18:05:17.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.852+0000 7f03a6b86980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 : hit (requested=0x17, cached=0x17) 2024-03-04T18:05:17.851 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.852+0000 7f03a6b86980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 2024-03-04T18:05:17.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.856+0000 7f03a6b86980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:05:17.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.856+0000 7f03a6b86980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:05:17.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.856+0000 7f03a6b86980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:17.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.856+0000 7f03a6b86980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23, ofs0, ns] 2024-03-04T18:05:17.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.856+0000 7f027affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595114 cookie 94673725345920 notifier 6194 bl.length()=845 2024-03-04T18:05:17.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.856+0000 7f027affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 info.flags=0x17 2024-03-04T18:05:17.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.856+0000 7f027affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:797b67a4-b876-4d04-b122-10afd858d465.4453.23 to cache LRU end 2024-03-04T18:05:17.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.856+0000 7f027affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:17.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.860+0000 7f03a6b86980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_sources= new_sources= 2024-03-04T18:05:17.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.860+0000 7f03a6b86980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential sources added=[] removed=[] 2024-03-04T18:05:17.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.860+0000 7f03a6b86980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): orig_dests= new_dests= 2024-03-04T18:05:17.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.860+0000 7f03a6b86980 20 handle_bi_update(): bucket=:abort-at-block-writes[797b67a4-b876-4d04-b122-10afd858d465.4453.23]): potential dests added=[] removed=[] 2024-03-04T18:05:17.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.864+0000 7f03a6b86980 1 execute INFO: reshard of bucket "abort-at-block-writes" completed successfully 2024-03-04T18:05:17.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.868+0000 7f025afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:17.869 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.872+0000 7f03a6b86980 20 remove_watcher() i=0 2024-03-04T18:05:17.869 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.872+0000 7f03a6b86980 2 removed watcher, disabling cache 2024-03-04T18:05:17.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.872+0000 7f03a6b86980 20 remove_watcher() i=1 2024-03-04T18:05:17.874 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.876+0000 7f03a6b86980 20 remove_watcher() i=2 2024-03-04T18:05:17.876 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.880+0000 7f03a6b86980 20 remove_watcher() i=3 2024-03-04T18:05:17.878 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.880+0000 7f03a6b86980 20 remove_watcher() i=4 2024-03-04T18:05:17.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.884+0000 7f03a6b86980 20 remove_watcher() i=5 2024-03-04T18:05:17.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.884+0000 7f03a6b86980 20 remove_watcher() i=6 2024-03-04T18:05:17.885 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.888+0000 7f03a6b86980 20 remove_watcher() i=7 2024-03-04T18:05:17.904 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-03-04T18:05:17.935 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:17.935 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:17.981 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.984+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.984+0000 7f69a3a0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.984+0000 7f69a3a0c980 20 realm 2024-03-04T18:05:17.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.984+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.984+0000 7f69a3a0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.984+0000 7f69a3a0c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:17.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.984+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.988+0000 7f69a3a0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:17.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.988+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.988+0000 7f69a3a0c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:17.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.988+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:17.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.988+0000 7f69a3a0c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:17.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.988+0000 7f69a3a0c980 20 searching for the correct realm 2024-03-04T18:05:17.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:17.992+0000 7f69a1524640 0 -- 172.21.15.47:0/899894972 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f699c10fc20 msgr2=0x7f699c0bb750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:18.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.204+0000 7f69a1524640 0 -- 172.21.15.47:0/899894972 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f6994029040 msgr2=0x7f6994002410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:18.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:18.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:18.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:18.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:18.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:18.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:18.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:18.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:18.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:18.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 zone default found 2024-03-04T18:05:18.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 4 Realm: () 2024-03-04T18:05:18.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:18.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:18.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:18.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 zonegroup default 2024-03-04T18:05:18.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.424+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.428+0000 7f69a3a0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:18.424 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.428+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.428+0000 7f69a3a0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:18.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.428+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.428+0000 7f69a3a0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:18.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.428+0000 7f69a3a0c980 20 started sync module instance, tier type = 2024-03-04T18:05:18.426 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.428+0000 7f69a3a0c980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:18.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.452+0000 7f69a3a0c980 20 add_watcher() i=0 2024-03-04T18:05:18.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.452+0000 7f69a3a0c980 20 add_watcher() i=1 2024-03-04T18:05:18.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.452+0000 7f69a3a0c980 20 add_watcher() i=2 2024-03-04T18:05:18.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.452+0000 7f69a3a0c980 20 add_watcher() i=3 2024-03-04T18:05:18.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.452+0000 7f69a3a0c980 20 add_watcher() i=4 2024-03-04T18:05:18.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.452+0000 7f69a3a0c980 20 add_watcher() i=5 2024-03-04T18:05:18.451 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.452+0000 7f69a3a0c980 20 add_watcher() i=6 2024-03-04T18:05:18.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.456+0000 7f69a3a0c980 20 add_watcher() i=7 2024-03-04T18:05:18.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.456+0000 7f69a3a0c980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:18.458 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.460+0000 7f6857fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:18.459 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.460+0000 7f69a3a0c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:18.459 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.460+0000 7f69a3a0c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:18.459 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.460+0000 7f69a3a0c980 5 note: GC not initialized 2024-03-04T18:05:18.459 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.460+0000 7f6856fe5640 20 reqs_thread_entry: start 2024-03-04T18:05:18.459 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.460+0000 7f69a3a0c980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:18.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.460+0000 7f69a3a0c980 20 Filter name: none 2024-03-04T18:05:18.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.464+0000 7f6854fe1640 20 reqs_thread_entry: start 2024-03-04T18:05:18.460 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.464+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.464+0000 7f69a3a0c980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:05:18.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.464+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.464+0000 7f69a3a0c980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:05:18.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.464+0000 7f69a3a0c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:18.463 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.464+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.468+0000 7f69a3a0c980 20 rados_obj.operate() r=0 bl.length=189 2024-03-04T18:05:18.464 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.468+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.468+0000 7f69a3a0c980 20 rados_obj.operate() r=0 bl.length=373 2024-03-04T18:05:18.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.468+0000 7f69a3a0c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:18.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.468+0000 7f69a3a0c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:18.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.468+0000 7f69a3a0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:18.469 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.472+0000 7f6857fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:18.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.476+0000 7f69a3a0c980 20 remove_watcher() i=0 2024-03-04T18:05:18.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.476+0000 7f69a3a0c980 2 removed watcher, disabling cache 2024-03-04T18:05:18.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.476+0000 7f69a3a0c980 20 remove_watcher() i=1 2024-03-04T18:05:18.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.480+0000 7f69a3a0c980 20 remove_watcher() i=2 2024-03-04T18:05:18.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.480+0000 7f69a3a0c980 20 remove_watcher() i=3 2024-03-04T18:05:18.481 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.484+0000 7f69a3a0c980 20 remove_watcher() i=4 2024-03-04T18:05:18.483 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.484+0000 7f69a3a0c980 20 remove_watcher() i=5 2024-03-04T18:05:18.486 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.488+0000 7f69a3a0c980 20 remove_watcher() i=6 2024-03-04T18:05:18.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:18.492+0000 7f69a3a0c980 20 remove_watcher() i=7 2024-03-04T18:05:18.506 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:18.506 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:05:18.506 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-block-writes id: 797b67a4-b876-4d04-b122-10afd858d465.4453.23 num_objects: 19 size_kb: 1 num_shards: 12 2024-03-04T18:05:18.506 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:18.508 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:18.508 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:18.531 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:18.531 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:18.936 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:18.936 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:19.314 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with EIO injected at commit_target_layout 2024-03-04T18:05:19.314 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:19.317 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:19.317 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:19.447 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:19.447 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:19.532 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:19.532 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:19.619 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:19.619 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:19.707 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:19.707 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:19.794 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:19.794 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:19.881 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:19.881 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:19.966 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:19.966 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.049 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.049 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.134 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.134 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.218 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.218 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.303 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.303 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.390 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.390 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.473 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.473 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.555 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.556 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.647 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.647 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.752 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.753 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.839 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.839 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:20.924 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:20.924 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:21.009 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:21.009 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:21.095 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:21.095 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:21.184 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:21.184 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:21.265 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:05:21.299 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:21.299 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:21.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.344+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.346 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.348+0000 7f58c82e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.348+0000 7f58c82e4980 20 realm 2024-03-04T18:05:21.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.348+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.348+0000 7f58c82e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.348+0000 7f58c82e4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:21.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.348+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.348+0000 7f58c82e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.348+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.352+0000 7f58c82e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:21.348 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.352+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.352+0000 7f58c82e4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:21.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.352+0000 7f58c82e4980 20 searching for the correct realm 2024-03-04T18:05:21.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.360+0000 7f58c5dfc640 0 -- 172.21.15.47:0/3866403738 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f58c004f040 msgr2=0x7f58c0035430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:21.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.364+0000 7f58c5dfc640 0 -- 172.21.15.47:0/3866403738 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x55a406ac6160 msgr2=0x55a406ac8620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:21.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:21.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:21.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:21.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:21.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:21.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:21.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 zone default found 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 4 Realm: () 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:21.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:21.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 zonegroup default 2024-03-04T18:05:21.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.372+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.376+0000 7f58c82e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.376+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.376+0000 7f58c82e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.376+0000 7f58c82e4980 20 started sync module instance, tier type = 2024-03-04T18:05:21.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.376+0000 7f58c82e4980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:21.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.396+0000 7f58c82e4980 20 add_watcher() i=0 2024-03-04T18:05:21.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.396+0000 7f58c82e4980 20 add_watcher() i=1 2024-03-04T18:05:21.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.396+0000 7f58c82e4980 20 add_watcher() i=2 2024-03-04T18:05:21.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.396+0000 7f58c82e4980 20 add_watcher() i=3 2024-03-04T18:05:21.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.396+0000 7f58c82e4980 20 add_watcher() i=4 2024-03-04T18:05:21.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.396+0000 7f58c82e4980 20 add_watcher() i=5 2024-03-04T18:05:21.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.396+0000 7f58c82e4980 20 add_watcher() i=6 2024-03-04T18:05:21.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.396+0000 7f58c82e4980 20 add_watcher() i=7 2024-03-04T18:05:21.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.396+0000 7f58c82e4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:21.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.400+0000 7f57777fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:21.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.400+0000 7f58c82e4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:21.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.400+0000 7f58c82e4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:21.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.400+0000 7f58c82e4980 5 note: GC not initialized 2024-03-04T18:05:21.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.400+0000 7f57767fc640 20 reqs_thread_entry: start 2024-03-04T18:05:21.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.400+0000 7f58c82e4980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:21.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.400+0000 7f58c82e4980 20 Filter name: none 2024-03-04T18:05:21.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.400+0000 7f56e7fff640 20 reqs_thread_entry: start 2024-03-04T18:05:21.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:21.401 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:21.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:21.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:21.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:21.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:21.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.404+0000 7f58c82e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.404 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.408+0000 7f58c82e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.408+0000 7f57777fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:21.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.412+0000 7f58c82e4980 20 remove_watcher() i=0 2024-03-04T18:05:21.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.412+0000 7f58c82e4980 2 removed watcher, disabling cache 2024-03-04T18:05:21.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.412+0000 7f58c82e4980 20 remove_watcher() i=1 2024-03-04T18:05:21.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.416+0000 7f58c82e4980 20 remove_watcher() i=2 2024-03-04T18:05:21.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.416+0000 7f58c82e4980 20 remove_watcher() i=3 2024-03-04T18:05:21.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.420+0000 7f58c82e4980 20 remove_watcher() i=4 2024-03-04T18:05:21.419 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.420+0000 7f58c82e4980 20 remove_watcher() i=5 2024-03-04T18:05:21.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.424+0000 7f58c82e4980 20 remove_watcher() i=6 2024-03-04T18:05:21.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.428+0000 7f58c82e4980 20 remove_watcher() i=7 2024-03-04T18:05:21.439 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:21.439 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:05:21.440 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-commit-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.24 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:05:21.440 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:21.440 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout 2024-03-04T18:05:21.472 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:21.472 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:21.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.520+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.520+0000 7f8a86053980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.520+0000 7f8a86053980 20 realm 2024-03-04T18:05:21.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.520+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.524+0000 7f8a86053980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.524+0000 7f8a86053980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:21.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.524+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.524+0000 7f8a86053980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.524+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.524+0000 7f8a86053980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:21.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.524+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.524+0000 7f8a86053980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:21.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.524+0000 7f8a86053980 20 searching for the correct realm 2024-03-04T18:05:21.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.528+0000 7f8a83b6b640 0 -- 172.21.15.47:0/2369273719 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f8a74082030 msgr2=0x7f8a74053ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:21.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.740+0000 7f8a83b6b640 0 -- 172.21.15.47:0/2369273719 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f8a7c0c1040 msgr2=0x7f8a7c0c1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:21.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.956+0000 7f8a86053980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:21.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.956+0000 7f8a86053980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:21.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.956+0000 7f8a86053980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:21.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.956+0000 7f8a86053980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:21.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.956+0000 7f8a86053980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:21.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.956+0000 7f8a86053980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:21.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.956+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:21.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:21.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 zone default found 2024-03-04T18:05:21.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 4 Realm: () 2024-03-04T18:05:21.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:21.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:21.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:21.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 zonegroup default 2024-03-04T18:05:21.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.960+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.964+0000 7f8a86053980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.964+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.964+0000 7f8a86053980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:21.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.964+0000 7f8a86053980 20 started sync module instance, tier type = 2024-03-04T18:05:21.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.964+0000 7f8a86053980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:21.985 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.988+0000 7f8a86053980 20 add_watcher() i=0 2024-03-04T18:05:21.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.988+0000 7f8a86053980 20 add_watcher() i=1 2024-03-04T18:05:21.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.988+0000 7f8a86053980 20 add_watcher() i=2 2024-03-04T18:05:21.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.988+0000 7f8a86053980 20 add_watcher() i=3 2024-03-04T18:05:21.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.988+0000 7f8a86053980 20 add_watcher() i=4 2024-03-04T18:05:21.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.988+0000 7f8a86053980 20 add_watcher() i=5 2024-03-04T18:05:21.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.988+0000 7f8a86053980 20 add_watcher() i=6 2024-03-04T18:05:21.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.992+0000 7f8a86053980 20 add_watcher() i=7 2024-03-04T18:05:21.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.992+0000 7f8a86053980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:21.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8939fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:21.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:21.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:21.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 5 note: GC not initialized 2024-03-04T18:05:21.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8938fe1640 20 reqs_thread_entry: start 2024-03-04T18:05:21.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:21.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 20 Filter name: none 2024-03-04T18:05:21.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f88b27fc640 20 reqs_thread_entry: start 2024-03-04T18:05:21.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-03-04T18:05:21.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:21.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-03-04T18:05:21.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-03-04T18:05:21.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 : miss 2024-03-04T18:05:21.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:21.996+0000 7f8a86053980 20 rados->read ofs=0 len=0 2024-03-04T18:05:21.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.000+0000 7f8a86053980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:21.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.000+0000 7f8a86053980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 info.flags=0x17 2024-03-04T18:05:21.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.000+0000 7f8a86053980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 to cache LRU end 2024-03-04T18:05:21.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.000+0000 7f8a86053980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:21.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.000+0000 7f8a86053980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:21.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.000+0000 7f8a86053980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 2024-03-04T18:05:22.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.012+0000 7f8a86053980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 info.flags=0x17 2024-03-04T18:05:22.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.012+0000 7f8a86053980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 to cache LRU end 2024-03-04T18:05:22.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.012+0000 7f8a86053980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:22.008 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.012+0000 7f8a86053980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24, ofs0, ns] 2024-03-04T18:05:22.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.012+0000 7f8962ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595104 cookie 94863228077152 notifier 6630 bl.length()=893 2024-03-04T18:05:22.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.012+0000 7f8962ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 info.flags=0x17 2024-03-04T18:05:22.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.012+0000 7f8962ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 to cache LRU end 2024-03-04T18:05:22.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.012+0000 7f8962ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:22.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.016+0000 7f8a86053980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): orig_sources= new_sources= 2024-03-04T18:05:22.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.016+0000 7f8a86053980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): potential sources added=[] removed=[] 2024-03-04T18:05:22.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.016+0000 7f8a86053980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): orig_dests= new_dests= 2024-03-04T18:05:22.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.016+0000 7f8a86053980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): potential dests added=[] removed=[] 2024-03-04T18:05:22.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.0 2024-03-04T18:05:22.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.0 2024-03-04T18:05:22.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.1 2024-03-04T18:05:22.018 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.1 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.2 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.2 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.3 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.3 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.4 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.4 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.5 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.5 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.6 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.6 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.7 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.7 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.8 2024-03-04T18:05:22.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.8 2024-03-04T18:05:22.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.9 2024-03-04T18:05:22.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.9 2024-03-04T18:05:22.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.10 2024-03-04T18:05:22.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.10 2024-03-04T18:05:22.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.11 2024-03-04T18:05:22.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1.11 2024-03-04T18:05:22.020 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.0 2024-03-04T18:05:22.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.020+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1 2024-03-04T18:05:22.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.024+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2 2024-03-04T18:05:22.022 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.024+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.3 2024-03-04T18:05:22.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.024+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.4 2024-03-04T18:05:22.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.024+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.5 2024-03-04T18:05:22.024 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.028+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.6 2024-03-04T18:05:22.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.028+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.7 2024-03-04T18:05:22.026 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.028+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.8 2024-03-04T18:05:22.027 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.028+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.9 2024-03-04T18:05:22.029 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.032+0000 7f8a86053980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.10 2024-03-04T18:05:22.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.036+0000 7f8a86053980 -1 Injecting error=-5 at location=commit_target_layout 2024-03-04T18:05:22.036 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.036+0000 7f8a86053980 0 ERROR: commit_reshard failed to commit target index layout: (5) Input/output error 2024-03-04T18:05:22.046 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.048+0000 7f8939fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:22.050 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.052+0000 7f8a86053980 20 remove_watcher() i=0 2024-03-04T18:05:22.050 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.052+0000 7f8a86053980 2 removed watcher, disabling cache 2024-03-04T18:05:22.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.056+0000 7f8a86053980 20 remove_watcher() i=1 2024-03-04T18:05:22.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.056+0000 7f8a86053980 20 remove_watcher() i=2 2024-03-04T18:05:22.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.060+0000 7f8a86053980 20 remove_watcher() i=3 2024-03-04T18:05:22.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.064+0000 7f8a86053980 20 remove_watcher() i=4 2024-03-04T18:05:22.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.064+0000 7f8a86053980 20 remove_watcher() i=5 2024-03-04T18:05:22.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.068+0000 7f8a86053980 20 remove_watcher() i=6 2024-03-04T18:05:22.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.072+0000 7f8a86053980 20 remove_watcher() i=7 2024-03-04T18:05:22.085 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:05:22.112 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:22.113 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:22.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.160+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.160+0000 7fb677afb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.160+0000 7fb677afb980 20 realm 2024-03-04T18:05:22.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.160+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.160+0000 7fb677afb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.160+0000 7fb677afb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:22.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.160+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.164+0000 7fb677afb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.164+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.164+0000 7fb677afb980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:22.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.164+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.164+0000 7fb677afb980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:22.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.164+0000 7fb677afb980 20 searching for the correct realm 2024-03-04T18:05:22.170 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.172+0000 7fb675613640 0 -- 172.21.15.47:0/690235178 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fb668063120 msgr2=0x7fb668049ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:22.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.176+0000 7fb675613640 0 -- 172.21.15.47:0/690235178 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x558cb8484830 msgr2=0x558cb8486c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:22.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:22.182 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:22.183 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:22.183 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:22.183 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:22.183 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:22.183 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:22.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:22.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 zone default found 2024-03-04T18:05:22.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 4 Realm: () 2024-03-04T18:05:22.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:22.184 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:22.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:22.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 zonegroup default 2024-03-04T18:05:22.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.184+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.188+0000 7fb677afb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.185 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.188+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.188+0000 7fb677afb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.188+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.188+0000 7fb677afb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.188+0000 7fb677afb980 20 started sync module instance, tier type = 2024-03-04T18:05:22.186 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.188+0000 7fb677afb980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:22.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.204+0000 7fb677afb980 20 add_watcher() i=0 2024-03-04T18:05:22.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.204+0000 7fb677afb980 20 add_watcher() i=1 2024-03-04T18:05:22.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.204+0000 7fb677afb980 20 add_watcher() i=2 2024-03-04T18:05:22.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.204+0000 7fb677afb980 20 add_watcher() i=3 2024-03-04T18:05:22.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.204+0000 7fb677afb980 20 add_watcher() i=4 2024-03-04T18:05:22.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.204+0000 7fb677afb980 20 add_watcher() i=5 2024-03-04T18:05:22.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.208+0000 7fb677afb980 20 add_watcher() i=6 2024-03-04T18:05:22.206 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.208+0000 7fb677afb980 20 add_watcher() i=7 2024-03-04T18:05:22.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.208+0000 7fb677afb980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:22.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb52bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:22.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb677afb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:22.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb677afb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:22.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb677afb980 5 note: GC not initialized 2024-03-04T18:05:22.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb52afe5640 20 reqs_thread_entry: start 2024-03-04T18:05:22.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb677afb980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:22.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb677afb980 20 Filter name: none 2024-03-04T18:05:22.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb528fe1640 20 reqs_thread_entry: start 2024-03-04T18:05:22.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb677afb980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:22.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.212+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.216+0000 7fb677afb980 20 rados_obj.operate() r=0 bl.length=413 2024-03-04T18:05:22.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.216+0000 7fb677afb980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:22.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.216+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.216+0000 7fb677afb980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:22.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.216+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.216+0000 7fb677afb980 20 rados_obj.operate() r=0 bl.length=413 2024-03-04T18:05:22.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.216+0000 7fb677afb980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:22.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.216+0000 7fb677afb980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.216+0000 7fb677afb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.220+0000 7fb52bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:22.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.224+0000 7fb677afb980 20 remove_watcher() i=0 2024-03-04T18:05:22.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.224+0000 7fb677afb980 2 removed watcher, disabling cache 2024-03-04T18:05:22.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.224+0000 7fb677afb980 20 remove_watcher() i=1 2024-03-04T18:05:22.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.228+0000 7fb677afb980 20 remove_watcher() i=2 2024-03-04T18:05:22.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.228+0000 7fb677afb980 20 remove_watcher() i=3 2024-03-04T18:05:22.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.228+0000 7fb677afb980 20 remove_watcher() i=4 2024-03-04T18:05:22.230 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.232+0000 7fb677afb980 20 remove_watcher() i=5 2024-03-04T18:05:22.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.236+0000 7fb677afb980 20 remove_watcher() i=6 2024-03-04T18:05:22.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.236+0000 7fb677afb980 20 remove_watcher() i=7 2024-03-04T18:05:22.249 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:22.249 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:05:22.250 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-commit-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.24 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:05:22.250 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:22.251 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:22.251 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:22.285 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:22.286 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:22.358 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:05:22.394 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:22.394 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:22.438 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.440+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 realm 2024-03-04T18:05:22.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:22.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.442 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:22.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:22.444 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.444+0000 7fa60d37a980 20 searching for the correct realm 2024-03-04T18:05:22.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.452+0000 7fa60a691640 0 -- 172.21.15.47:0/3008044683 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fa5f402b030 msgr2=0x7fa5f404f000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:22.457 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.460+0000 7fa60b693640 0 -- 172.21.15.47:0/3008044683 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x56172ab4ec20 msgr2=0x56172ab510e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:22.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:22.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:22.465 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:22.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:22.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:22.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:22.466 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 zone default found 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 4 Realm: () 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 zonegroup default 2024-03-04T18:05:22.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.468+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.472+0000 7fa60d37a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.472+0000 7fa60d37a980 20 started sync module instance, tier type = 2024-03-04T18:05:22.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.472+0000 7fa60d37a980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:22.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.488+0000 7fa60d37a980 20 add_watcher() i=0 2024-03-04T18:05:22.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.488+0000 7fa60d37a980 20 add_watcher() i=1 2024-03-04T18:05:22.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.488+0000 7fa60d37a980 20 add_watcher() i=2 2024-03-04T18:05:22.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.488+0000 7fa60d37a980 20 add_watcher() i=3 2024-03-04T18:05:22.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.488+0000 7fa60d37a980 20 add_watcher() i=4 2024-03-04T18:05:22.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.488+0000 7fa60d37a980 20 add_watcher() i=5 2024-03-04T18:05:22.487 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.488+0000 7fa60d37a980 20 add_watcher() i=6 2024-03-04T18:05:22.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.492+0000 7fa60d37a980 20 add_watcher() i=7 2024-03-04T18:05:22.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.492+0000 7fa60d37a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:22.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa4c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:22.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:22.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 5 note: GC not initialized 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa43bfff640 20 reqs_thread_entry: start 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 20 Filter name: none 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa439ffb640 20 reqs_thread_entry: start 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 : miss 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 20 rados_obj.operate() r=0 bl.length=413 2024-03-04T18:05:22.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 info.flags=0x17 2024-03-04T18:05:22.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 to cache LRU end 2024-03-04T18:05:22.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:22.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:22.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.496+0000 7fa60d37a980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 2024-03-04T18:05:22.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.500+0000 7fa60d37a980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-03-04T18:05:22.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.512+0000 7fa60d37a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 info.flags=0x17 2024-03-04T18:05:22.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.512+0000 7fa60d37a980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 to cache LRU end 2024-03-04T18:05:22.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.512+0000 7fa60d37a980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:22.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.512+0000 7fa60d37a980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24, ofs0, ns] 2024-03-04T18:05:22.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.512+0000 7fa4edffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595105 cookie 94657500792240 notifier 6275 bl.length()=893 2024-03-04T18:05:22.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.512+0000 7fa4edffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 info.flags=0x17 2024-03-04T18:05:22.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.512+0000 7fa4edffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 to cache LRU end 2024-03-04T18:05:22.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.512+0000 7fa4edffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:22.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.516+0000 7fa60d37a980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): orig_sources= new_sources= 2024-03-04T18:05:22.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.516+0000 7fa60d37a980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): potential sources added=[] removed=[] 2024-03-04T18:05:22.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.516+0000 7fa60d37a980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): orig_dests= new_dests= 2024-03-04T18:05:22.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.516+0000 7fa60d37a980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): potential dests added=[] removed=[] 2024-03-04T18:05:22.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.520+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.0 2024-03-04T18:05:22.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.520+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.0 2024-03-04T18:05:22.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.1 2024-03-04T18:05:22.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.1 2024-03-04T18:05:22.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.2 2024-03-04T18:05:22.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.2 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.3 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.3 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.4 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.4 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.5 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.5 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.6 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.6 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.7 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.7 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.8 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.8 2024-03-04T18:05:22.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.9 2024-03-04T18:05:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.9 2024-03-04T18:05:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.10 2024-03-04T18:05:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.10 2024-03-04T18:05:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.11 2024-03-04T18:05:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2.11 2024-03-04T18:05:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.0 2024-03-04T18:05:22.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.1 2024-03-04T18:05:22.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.524+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.2 2024-03-04T18:05:22.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.528+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.3 2024-03-04T18:05:22.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.528+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.4 2024-03-04T18:05:22.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.528+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.5 2024-03-04T18:05:22.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.528+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.6 2024-03-04T18:05:22.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.528+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.7 2024-03-04T18:05:22.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.528+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.8 2024-03-04T18:05:22.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.532+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.9 2024-03-04T18:05:22.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.532+0000 7fa60d37a980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.24.10 2024-03-04T18:05:22.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa60d37a980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 : hit (requested=0x17, cached=0x17) 2024-03-04T18:05:22.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa60d37a980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 2024-03-04T18:05:22.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa60d37a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 info.flags=0x17 2024-03-04T18:05:22.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa60d37a980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 to cache LRU end 2024-03-04T18:05:22.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa60d37a980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:22.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa60d37a980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24, ofs0, ns] 2024-03-04T18:05:22.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa4ee7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595106 cookie 94657500792240 notifier 6275 bl.length()=861 2024-03-04T18:05:22.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa4ee7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 info.flags=0x17 2024-03-04T18:05:22.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa4ee7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.24 to cache LRU end 2024-03-04T18:05:22.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.540+0000 7fa4ee7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:22.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.544+0000 7fa60d37a980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): orig_sources= new_sources= 2024-03-04T18:05:22.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.544+0000 7fa60d37a980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): potential sources added=[] removed=[] 2024-03-04T18:05:22.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.544+0000 7fa60d37a980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): orig_dests= new_dests= 2024-03-04T18:05:22.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.544+0000 7fa60d37a980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.24]): potential dests added=[] removed=[] 2024-03-04T18:05:22.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.552+0000 7fa60d37a980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-03-04T18:05:22.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.552+0000 7fa4c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:22.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.560+0000 7fa60d37a980 20 remove_watcher() i=0 2024-03-04T18:05:22.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.560+0000 7fa60d37a980 2 removed watcher, disabling cache 2024-03-04T18:05:22.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.560+0000 7fa60d37a980 20 remove_watcher() i=1 2024-03-04T18:05:22.561 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.564+0000 7fa60d37a980 20 remove_watcher() i=2 2024-03-04T18:05:22.563 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.564+0000 7fa60d37a980 20 remove_watcher() i=3 2024-03-04T18:05:22.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.568+0000 7fa60d37a980 20 remove_watcher() i=4 2024-03-04T18:05:22.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.572+0000 7fa60d37a980 20 remove_watcher() i=5 2024-03-04T18:05:22.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.572+0000 7fa60d37a980 20 remove_watcher() i=6 2024-03-04T18:05:22.573 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.576+0000 7fa60d37a980 20 remove_watcher() i=7 2024-03-04T18:05:22.590 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:05:22.617 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:22.617 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:22.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.664+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.664+0000 7f3df7d12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.664+0000 7f3df7d12980 20 realm 2024-03-04T18:05:22.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.664+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.664+0000 7f3df7d12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.664+0000 7f3df7d12980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:22.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.664+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.664+0000 7f3df7d12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.664+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.668+0000 7f3df7d12980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:22.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.668+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.668+0000 7f3df7d12980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:22.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.668+0000 7f3df7d12980 20 searching for the correct realm 2024-03-04T18:05:22.672 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.676+0000 7f3df5029640 0 -- 172.21.15.47:0/1699750601 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f3de0038400 msgr2=0x7f3de0038880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:22.678 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.680+0000 7f3df602b640 0 -- 172.21.15.47:0/1699750601 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55f729f669e0 msgr2=0x55f729f68ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:22.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:22.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:22.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:22.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:22.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:22.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:22.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 zone default found 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 4 Realm: () 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:22.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:22.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 zonegroup default 2024-03-04T18:05:22.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.688+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.692+0000 7f3df7d12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.692+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.689 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.692+0000 7f3df7d12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.689 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.692+0000 7f3df7d12980 20 started sync module instance, tier type = 2024-03-04T18:05:22.689 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.692+0000 7f3df7d12980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:22.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.708+0000 7f3df7d12980 20 add_watcher() i=0 2024-03-04T18:05:22.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.708+0000 7f3df7d12980 20 add_watcher() i=1 2024-03-04T18:05:22.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.708+0000 7f3df7d12980 20 add_watcher() i=2 2024-03-04T18:05:22.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.708+0000 7f3df7d12980 20 add_watcher() i=3 2024-03-04T18:05:22.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.708+0000 7f3df7d12980 20 add_watcher() i=4 2024-03-04T18:05:22.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.708+0000 7f3df7d12980 20 add_watcher() i=5 2024-03-04T18:05:22.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.708+0000 7f3df7d12980 20 add_watcher() i=6 2024-03-04T18:05:22.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.712+0000 7f3df7d12980 20 add_watcher() i=7 2024-03-04T18:05:22.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.712+0000 7f3df7d12980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:22.712 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3c977fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:22.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:22.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:22.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 5 note: GC not initialized 2024-03-04T18:05:22.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3c967fc640 20 reqs_thread_entry: start 2024-03-04T18:05:22.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:22.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 Filter name: none 2024-03-04T18:05:22.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3c17fff640 20 reqs_thread_entry: start 2024-03-04T18:05:22.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:22.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:22.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:22.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:22.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:22.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.716+0000 7f3df7d12980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:22.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.720+0000 7f3df7d12980 20 rados->read ofs=0 len=0 2024-03-04T18:05:22.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.720+0000 7f3df7d12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:22.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.720+0000 7f3c977fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:22.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.724+0000 7f3df7d12980 20 remove_watcher() i=0 2024-03-04T18:05:22.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.724+0000 7f3df7d12980 2 removed watcher, disabling cache 2024-03-04T18:05:22.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.728+0000 7f3df7d12980 20 remove_watcher() i=1 2024-03-04T18:05:22.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.728+0000 7f3df7d12980 20 remove_watcher() i=2 2024-03-04T18:05:22.729 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.732+0000 7f3df7d12980 20 remove_watcher() i=3 2024-03-04T18:05:22.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.732+0000 7f3df7d12980 20 remove_watcher() i=4 2024-03-04T18:05:22.733 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.736+0000 7f3df7d12980 20 remove_watcher() i=5 2024-03-04T18:05:22.736 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.736+0000 7f3df7d12980 20 remove_watcher() i=6 2024-03-04T18:05:22.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:22.740+0000 7f3df7d12980 20 remove_watcher() i=7 2024-03-04T18:05:22.753 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:22.753 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:05:22.753 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-commit-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.24 num_objects: 19 size_kb: 1 num_shards: 12 2024-03-04T18:05:22.753 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:22.754 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:22.754 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:22.773 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:22.773 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:23.081 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:23.081 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:23.461 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with ECANCELED injected at commit_target_layout 2024-03-04T18:05:23.461 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:23.463 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:23.463 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:23.613 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:23.613 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:23.695 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:23.696 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:23.781 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:23.781 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:23.867 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:23.867 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:23.952 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:23.952 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.049 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.049 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.132 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.132 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.225 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.225 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.313 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.313 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.398 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.398 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.487 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.487 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.570 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.570 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.656 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.656 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.741 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.741 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.829 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.829 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:24.912 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:24.912 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:25.003 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:25.003 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:25.093 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:25.093 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:25.184 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:25.184 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:25.272 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:25.273 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:25.361 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:25.361 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:25.496 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:05:25.533 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:25.534 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:25.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.579+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 realm 2024-03-04T18:05:25.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:25.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:25.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:25.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.583+0000 7f0b74787980 20 searching for the correct realm 2024-03-04T18:05:25.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.591+0000 7f0b72aa0640 0 -- 172.21.15.47:0/1386853466 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f0b68027020 msgr2=0x7f0b68074aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:25.595 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.595+0000 7f0b7229f640 0 -- 172.21.15.47:0/1386853466 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f0b6c018050 msgr2=0x7f0b6c03d410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:25.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:25.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:25.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:25.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:25.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:25.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:25.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:25.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.603+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:25.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 zone default found 2024-03-04T18:05:25.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 4 Realm: () 2024-03-04T18:05:25.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:25.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:25.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:25.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 zonegroup default 2024-03-04T18:05:25.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 started sync module instance, tier type = 2024-03-04T18:05:25.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.607+0000 7f0b74787980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:25.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.627+0000 7f0b74787980 20 add_watcher() i=0 2024-03-04T18:05:25.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.627+0000 7f0b74787980 20 add_watcher() i=1 2024-03-04T18:05:25.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.627+0000 7f0b74787980 20 add_watcher() i=2 2024-03-04T18:05:25.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.627+0000 7f0b74787980 20 add_watcher() i=3 2024-03-04T18:05:25.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.627+0000 7f0b74787980 20 add_watcher() i=4 2024-03-04T18:05:25.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.627+0000 7f0b74787980 20 add_watcher() i=5 2024-03-04T18:05:25.626 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.627+0000 7f0b74787980 20 add_watcher() i=6 2024-03-04T18:05:25.627 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.627+0000 7f0b74787980 20 add_watcher() i=7 2024-03-04T18:05:25.627 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.627+0000 7f0b74787980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:25.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.631+0000 7f0a28fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:25.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.631+0000 7f0b74787980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:25.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.631+0000 7f0b74787980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:25.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.631+0000 7f0b74787980 5 note: GC not initialized 2024-03-04T18:05:25.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f09a37fe640 20 reqs_thread_entry: start 2024-03-04T18:05:25.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:25.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 Filter name: none 2024-03-04T18:05:25.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f09a17fa640 20 reqs_thread_entry: start 2024-03-04T18:05:25.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:25.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:25.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:25.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:25.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:25.635 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.635+0000 7f0b74787980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:25.636 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.639+0000 7f0b74787980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.639+0000 7f0b74787980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.639+0000 7f0a28fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:25.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.643+0000 7f0b74787980 20 remove_watcher() i=0 2024-03-04T18:05:25.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.643+0000 7f0b74787980 2 removed watcher, disabling cache 2024-03-04T18:05:25.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.647+0000 7f0b74787980 20 remove_watcher() i=1 2024-03-04T18:05:25.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.647+0000 7f0b74787980 20 remove_watcher() i=2 2024-03-04T18:05:25.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.651+0000 7f0b74787980 20 remove_watcher() i=3 2024-03-04T18:05:25.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.651+0000 7f0b74787980 20 remove_watcher() i=4 2024-03-04T18:05:25.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.655+0000 7f0b74787980 20 remove_watcher() i=5 2024-03-04T18:05:25.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.655+0000 7f0b74787980 20 remove_watcher() i=6 2024-03-04T18:05:25.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.659+0000 7f0b74787980 20 remove_watcher() i=7 2024-03-04T18:05:25.672 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:25.672 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:05:25.672 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-commit-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.25 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:05:25.673 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:25.673 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout --inject-error-code 125 2024-03-04T18:05:25.709 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:25.709 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:25.754 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.755+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 realm 2024-03-04T18:05:25.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.756 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:25.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:25.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:25.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.759+0000 7f06458e4980 20 searching for the correct realm 2024-03-04T18:05:25.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.767+0000 7f06433fc640 0 -- 172.21.15.47:0/2205634893 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f06340c6040 msgr2=0x7f06340ba440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:25.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.771+0000 7f06433fc640 0 -- 172.21.15.47:0/2205634893 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x5643ebbcc660 msgr2=0x5643ebbceab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:25.988 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:25.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:25.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:25.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:25.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:25.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:25.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:25.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:25.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 zone default found 2024-03-04T18:05:25.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 4 Realm: () 2024-03-04T18:05:25.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:25.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:25.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:25.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 zonegroup default 2024-03-04T18:05:25.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.991+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.995+0000 7f06458e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.995+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.995+0000 7f06458e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.995+0000 7f06458e4980 20 started sync module instance, tier type = 2024-03-04T18:05:25.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:25.995+0000 7f06458e4980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:26.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.075+0000 7f06458e4980 20 add_watcher() i=0 2024-03-04T18:05:26.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.075+0000 7f06458e4980 20 add_watcher() i=1 2024-03-04T18:05:26.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.075+0000 7f06458e4980 20 add_watcher() i=2 2024-03-04T18:05:26.073 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.075+0000 7f06458e4980 20 add_watcher() i=3 2024-03-04T18:05:26.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.075+0000 7f06458e4980 20 add_watcher() i=4 2024-03-04T18:05:26.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.075+0000 7f06458e4980 20 add_watcher() i=5 2024-03-04T18:05:26.074 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.075+0000 7f06458e4980 20 add_watcher() i=6 2024-03-04T18:05:26.076 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.079+0000 7f06458e4980 20 add_watcher() i=7 2024-03-04T18:05:26.076 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.079+0000 7f06458e4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:26.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.083+0000 7f04f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:26.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.083+0000 7f06458e4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:26.082 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.083+0000 7f06458e4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:26.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.083+0000 7f06458e4980 5 note: GC not initialized 2024-03-04T18:05:26.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.083+0000 7f04f8fe1640 20 reqs_thread_entry: start 2024-03-04T18:05:26.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.083+0000 7f06458e4980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:26.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.083+0000 7f06458e4980 20 Filter name: none 2024-03-04T18:05:26.084 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.083+0000 7f04727fc640 20 reqs_thread_entry: start 2024-03-04T18:05:26.084 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-03-04T18:05:26.084 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:26.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-03-04T18:05:26.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-03-04T18:05:26.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 : miss 2024-03-04T18:05:26.085 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:26.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 info.flags=0x17 2024-03-04T18:05:26.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 to cache LRU end 2024-03-04T18:05:26.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:26.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:26.086 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.087+0000 7f06458e4980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 2024-03-04T18:05:26.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.099+0000 7f06458e4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 info.flags=0x17 2024-03-04T18:05:26.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.099+0000 7f06458e4980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 to cache LRU end 2024-03-04T18:05:26.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.099+0000 7f06458e4980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:26.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.099+0000 7f06458e4980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25, ofs0, ns] 2024-03-04T18:05:26.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.099+0000 7f0526ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595109 cookie 94849717819488 notifier 6323 bl.length()=893 2024-03-04T18:05:26.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.099+0000 7f0526ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 info.flags=0x17 2024-03-04T18:05:26.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.099+0000 7f0526ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 to cache LRU end 2024-03-04T18:05:26.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.099+0000 7f0526ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:26.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.103+0000 7f06458e4980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.25]): orig_sources= new_sources= 2024-03-04T18:05:26.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.103+0000 7f06458e4980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.25]): potential sources added=[] removed=[] 2024-03-04T18:05:26.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.103+0000 7f06458e4980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.25]): orig_dests= new_dests= 2024-03-04T18:05:26.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.103+0000 7f06458e4980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.25]): potential dests added=[] removed=[] 2024-03-04T18:05:26.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.0 2024-03-04T18:05:26.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.0 2024-03-04T18:05:26.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.1 2024-03-04T18:05:26.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.1 2024-03-04T18:05:26.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.2 2024-03-04T18:05:26.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.2 2024-03-04T18:05:26.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.3 2024-03-04T18:05:26.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.3 2024-03-04T18:05:26.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.4 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.4 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.5 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.5 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.6 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.6 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.7 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.7 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.8 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.8 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.9 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.9 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.10 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.10 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.11 2024-03-04T18:05:26.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1.11 2024-03-04T18:05:26.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.107+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.0 2024-03-04T18:05:26.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.111+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.1 2024-03-04T18:05:26.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.111+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.2 2024-03-04T18:05:26.111 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.111+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.3 2024-03-04T18:05:26.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.111+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.4 2024-03-04T18:05:26.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.115+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.5 2024-03-04T18:05:26.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.115+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.6 2024-03-04T18:05:26.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.115+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.7 2024-03-04T18:05:26.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.115+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.8 2024-03-04T18:05:26.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.119+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.9 2024-03-04T18:05:26.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.119+0000 7f06458e4980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.25.10 2024-03-04T18:05:26.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f06458e4980 -1 Injecting error=-125 at location=commit_target_layout 2024-03-04T18:05:26.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f06458e4980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 : hit (requested=0x17, cached=0x17) 2024-03-04T18:05:26.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f06458e4980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 2024-03-04T18:05:26.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f06458e4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 info.flags=0x17 2024-03-04T18:05:26.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f06458e4980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 to cache LRU end 2024-03-04T18:05:26.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f06458e4980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:26.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f06458e4980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25, ofs0, ns] 2024-03-04T18:05:26.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f05267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595110 cookie 94849717819488 notifier 6323 bl.length()=861 2024-03-04T18:05:26.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f05267fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 info.flags=0x17 2024-03-04T18:05:26.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f05267fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.25 to cache LRU end 2024-03-04T18:05:26.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.127+0000 7f05267fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:26.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.131+0000 7f06458e4980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.25]): orig_sources= new_sources= 2024-03-04T18:05:26.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.131+0000 7f06458e4980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.25]): potential sources added=[] removed=[] 2024-03-04T18:05:26.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.131+0000 7f06458e4980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.25]): orig_dests= new_dests= 2024-03-04T18:05:26.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.131+0000 7f06458e4980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.25]): potential dests added=[] removed=[] 2024-03-04T18:05:26.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.135+0000 7f06458e4980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-03-04T18:05:26.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.143+0000 7f04f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:26.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.147+0000 7f06458e4980 20 remove_watcher() i=0 2024-03-04T18:05:26.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.147+0000 7f06458e4980 2 removed watcher, disabling cache 2024-03-04T18:05:26.148 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.147+0000 7f06458e4980 20 remove_watcher() i=1 2024-03-04T18:05:26.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.151+0000 7f06458e4980 20 remove_watcher() i=2 2024-03-04T18:05:26.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.155+0000 7f06458e4980 20 remove_watcher() i=3 2024-03-04T18:05:26.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.155+0000 7f06458e4980 20 remove_watcher() i=4 2024-03-04T18:05:26.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.159+0000 7f06458e4980 20 remove_watcher() i=5 2024-03-04T18:05:26.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.159+0000 7f06458e4980 20 remove_watcher() i=6 2024-03-04T18:05:26.162 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.163+0000 7f06458e4980 20 remove_watcher() i=7 2024-03-04T18:05:26.179 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:05:26.209 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:26.210 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:26.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.255+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.255+0000 7f5a902fa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.255+0000 7f5a902fa980 20 realm 2024-03-04T18:05:26.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.255+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.255+0000 7f5a902fa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.255+0000 7f5a902fa980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:26.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.255+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.259+0000 7f5a902fa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.259+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.259+0000 7f5a902fa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:26.257 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.259+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.259+0000 7f5a902fa980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:26.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.259+0000 7f5a902fa980 20 searching for the correct realm 2024-03-04T18:05:26.265 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.267+0000 7f5a8e613640 0 -- 172.21.15.47:0/1930891265 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f5a780c2020 msgr2=0x7f5a7810b8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:26.273 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.275+0000 7f5a8d611640 0 -- 172.21.15.47:0/1930891265 >> [v2:172.21.15.152:6808/929675582,v1:172.21.15.152:6809/929675582] conn(0x561e7c0ced90 msgr2=0x561e7c0d11c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:26.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:26.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:26.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:26.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:26.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:26.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:26.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:26.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:26.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 zone default found 2024-03-04T18:05:26.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 4 Realm: () 2024-03-04T18:05:26.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:26.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:26.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:26.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 zonegroup default 2024-03-04T18:05:26.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.475+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.479+0000 7f5a902fa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.479+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.479+0000 7f5a902fa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.479+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.479+0000 7f5a902fa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.479+0000 7f5a902fa980 20 started sync module instance, tier type = 2024-03-04T18:05:26.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.479+0000 7f5a902fa980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:26.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.503+0000 7f5a902fa980 20 add_watcher() i=0 2024-03-04T18:05:26.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.503+0000 7f5a902fa980 20 add_watcher() i=1 2024-03-04T18:05:26.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.503+0000 7f5a902fa980 20 add_watcher() i=2 2024-03-04T18:05:26.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.503+0000 7f5a902fa980 20 add_watcher() i=3 2024-03-04T18:05:26.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.503+0000 7f5a902fa980 20 add_watcher() i=4 2024-03-04T18:05:26.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.503+0000 7f5a902fa980 20 add_watcher() i=5 2024-03-04T18:05:26.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.503+0000 7f5a902fa980 20 add_watcher() i=6 2024-03-04T18:05:26.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.507+0000 7f5a902fa980 20 add_watcher() i=7 2024-03-04T18:05:26.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.507+0000 7f5a902fa980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:26.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.511+0000 7f592f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:26.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.511+0000 7f5a902fa980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:26.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.511+0000 7f5a902fa980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:26.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.511+0000 7f5a902fa980 5 note: GC not initialized 2024-03-04T18:05:26.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f592e7fc640 20 reqs_thread_entry: start 2024-03-04T18:05:26.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f5a902fa980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:26.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f5a902fa980 20 Filter name: none 2024-03-04T18:05:26.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f58b3fff640 20 reqs_thread_entry: start 2024-03-04T18:05:26.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f5a902fa980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:26.514 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f5a902fa980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:26.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f5a902fa980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:26.515 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.515+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.519+0000 7f5a902fa980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:26.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.519+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.519+0000 7f5a902fa980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:26.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.519+0000 7f5a902fa980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:26.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.519+0000 7f5a902fa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.519+0000 7f5a902fa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.523+0000 7f592f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:26.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.527+0000 7f5a902fa980 20 remove_watcher() i=0 2024-03-04T18:05:26.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.527+0000 7f5a902fa980 2 removed watcher, disabling cache 2024-03-04T18:05:26.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.527+0000 7f5a902fa980 20 remove_watcher() i=1 2024-03-04T18:05:26.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.531+0000 7f5a902fa980 20 remove_watcher() i=2 2024-03-04T18:05:26.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.531+0000 7f5a902fa980 20 remove_watcher() i=3 2024-03-04T18:05:26.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.535+0000 7f5a902fa980 20 remove_watcher() i=4 2024-03-04T18:05:26.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.539+0000 7f5a902fa980 20 remove_watcher() i=5 2024-03-04T18:05:26.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.539+0000 7f5a902fa980 20 remove_watcher() i=6 2024-03-04T18:05:26.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:26.543+0000 7f5a902fa980 20 remove_watcher() i=7 2024-03-04T18:05:26.559 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:26.559 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:05:26.559 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-commit-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.25 num_objects: 20 size_kb: 1 num_shards: 12 2024-03-04T18:05:26.560 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:26.561 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:26.561 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:26.579 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:26.579 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:27.049 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.050 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:27.424 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with abort at commit_target_layout 2024-03-04T18:05:27.424 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:27.427 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.427 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:27.557 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.557 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:27.640 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.641 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:27.729 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.730 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:27.824 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.824 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:27.910 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.910 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:27.995 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.995 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.082 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.082 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.169 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.169 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.255 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.255 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.340 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.340 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.424 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.424 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.510 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.510 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.600 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.600 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.689 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.689 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.771 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.771 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.857 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.857 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:28.941 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.941 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:29.025 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:29.025 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:29.110 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:29.111 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:29.200 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:29.200 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:29.288 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:29.288 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:05:29.368 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-03-04T18:05:29.403 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:29.403 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:29.449 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.451+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.451+0000 7f7b9f391980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.451+0000 7f7b9f391980 20 realm 2024-03-04T18:05:29.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.451+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.455+0000 7f7b9f391980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.455+0000 7f7b9f391980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:29.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.455+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.455+0000 7f7b9f391980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.455+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.455+0000 7f7b9f391980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:29.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.455+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.455+0000 7f7b9f391980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:29.454 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.455+0000 7f7b9f391980 20 searching for the correct realm 2024-03-04T18:05:29.461 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.463+0000 7f7b9d6aa640 0 -- 172.21.15.47:0/313869539 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f7b8803c040 msgr2=0x7f7b880309a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:29.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.467+0000 7f7b9cea9640 0 -- 172.21.15.47:0/313869539 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f7b98017020 msgr2=0x7f7b98071120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:29.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:29.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:29.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:29.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:29.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:29.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:29.476 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.477 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 zone default found 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 4 Realm: () 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 zonegroup default 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 started sync module instance, tier type = 2024-03-04T18:05:29.479 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.479+0000 7f7b9f391980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:29.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.499+0000 7f7b9f391980 20 add_watcher() i=0 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.499+0000 7f7b9f391980 20 add_watcher() i=1 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.499+0000 7f7b9f391980 20 add_watcher() i=2 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.499+0000 7f7b9f391980 20 add_watcher() i=3 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.499+0000 7f7b9f391980 20 add_watcher() i=4 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.499+0000 7f7b9f391980 20 add_watcher() i=5 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.499+0000 7f7b9f391980 20 add_watcher() i=6 2024-03-04T18:05:29.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.503+0000 7f7b9f391980 20 add_watcher() i=7 2024-03-04T18:05:29.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.503+0000 7f7b9f391980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:29.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.503+0000 7f7a537e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:29.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.503+0000 7f7b9f391980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:29.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.503+0000 7f7b9f391980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:29.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7b9f391980 5 note: GC not initialized 2024-03-04T18:05:29.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7a527e4640 20 reqs_thread_entry: start 2024-03-04T18:05:29.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7b9f391980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:29.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7b9f391980 20 Filter name: none 2024-03-04T18:05:29.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7a4b7fe640 20 reqs_thread_entry: start 2024-03-04T18:05:29.505 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7b9f391980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:29.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7b9f391980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:29.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7b9f391980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:29.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.507+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.511+0000 7f7b9f391980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:29.508 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.511+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.509 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.511+0000 7f7b9f391980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:29.509 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.511+0000 7f7b9f391980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:29.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.511+0000 7f7b9f391980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.511 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.511+0000 7f7b9f391980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.513 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.515+0000 7f7a537e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:29.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.519+0000 7f7b9f391980 20 remove_watcher() i=0 2024-03-04T18:05:29.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.519+0000 7f7b9f391980 2 removed watcher, disabling cache 2024-03-04T18:05:29.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.519+0000 7f7b9f391980 20 remove_watcher() i=1 2024-03-04T18:05:29.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.523+0000 7f7b9f391980 20 remove_watcher() i=2 2024-03-04T18:05:29.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.523+0000 7f7b9f391980 20 remove_watcher() i=3 2024-03-04T18:05:29.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.527+0000 7f7b9f391980 20 remove_watcher() i=4 2024-03-04T18:05:29.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.527+0000 7f7b9f391980 20 remove_watcher() i=5 2024-03-04T18:05:29.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.531+0000 7f7b9f391980 20 remove_watcher() i=6 2024-03-04T18:05:29.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.535+0000 7f7b9f391980 20 remove_watcher() i=7 2024-03-04T18:05:29.554 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:29.554 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:05:29.554 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-commit-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.26 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:05:29.554 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:29.554 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at commit_target_layout 2024-03-04T18:05:29.587 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:29.588 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:29.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.643+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 realm 2024-03-04T18:05:29.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:29.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:29.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:29.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 searching for the correct realm 2024-03-04T18:05:29.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.655+0000 7f69d6114640 0 -- 172.21.15.47:0/1258942298 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f69cc06a310 msgr2=0x7f69cc09a050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:29.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.867+0000 7f69d7116640 0 -- 172.21.15.47:0/1258942298 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55a7c7dafad0 msgr2=0x55a7c7db1f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:29.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:29.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:29.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:29.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:29.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:29.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:29.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.880 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:29.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:29.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 zone default found 2024-03-04T18:05:29.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 4 Realm: () 2024-03-04T18:05:29.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:29.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:29.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:29.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 zonegroup default 2024-03-04T18:05:29.882 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.883 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.887+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.887+0000 7f69d8dfd980 20 started sync module instance, tier type = 2024-03-04T18:05:29.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.887+0000 7f69d8dfd980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:29.906 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=0 2024-03-04T18:05:29.906 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=1 2024-03-04T18:05:29.906 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=2 2024-03-04T18:05:29.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=3 2024-03-04T18:05:29.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=4 2024-03-04T18:05:29.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=5 2024-03-04T18:05:29.907 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=6 2024-03-04T18:05:29.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.911+0000 7f69d8dfd980 20 add_watcher() i=7 2024-03-04T18:05:29.909 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.911+0000 7f69d8dfd980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:29.914 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.915+0000 7f688d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:29.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:29.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:29.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 note: GC not initialized 2024-03-04T18:05:29.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.915+0000 7f6807fff640 20 reqs_thread_entry: start 2024-03-04T18:05:29.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:29.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 Filter name: none 2024-03-04T18:05:29.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f6805ffb640 20 reqs_thread_entry: start 2024-03-04T18:05:29.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-03-04T18:05:29.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:29.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-03-04T18:05:29.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-03-04T18:05:29.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 : miss 2024-03-04T18:05:29.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:29.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:05:29.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:05:29.919 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:29.919 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:29.919 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 2024-03-04T18:05:29.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:05:29.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:05:29.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:29.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26, ofs0, ns] 2024-03-04T18:05:29.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595115 cookie 94179100970704 notifier 6371 bl.length()=893 2024-03-04T18:05:29.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:05:29.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:05:29.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:29.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_sources= new_sources= 2024-03-04T18:05:29.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential sources added=[] removed=[] 2024-03-04T18:05:29.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_dests= new_dests= 2024-03-04T18:05:29.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential dests added=[] removed=[] 2024-03-04T18:05:29.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.0 2024-03-04T18:05:29.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.0 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.1 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.1 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.2 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.2 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.3 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.3 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.4 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.4 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.5 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.5 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.6 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.6 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.7 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.7 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.8 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.8 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.9 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.9 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.10 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.10 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.11 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.11 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.0 2024-03-04T18:05:29.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1 2024-03-04T18:05:29.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.2 2024-03-04T18:05:29.943 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.3 2024-03-04T18:05:29.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.947+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.4 2024-03-04T18:05:29.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.947+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.5 2024-03-04T18:05:29.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.947+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.6 2024-03-04T18:05:29.948 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.7 2024-03-04T18:05:29.949 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.8 2024-03-04T18:05:29.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.9 2024-03-04T18:05:29.951 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.10 2024-03-04T18:05:29.959 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f69d8dfd980 time 2024-03-04T18:05:29.966607+0000 2024-03-04T18:05:29.959 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:05:29.961 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:29.961 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f69dc6d2d7b] 2024-03-04T18:05:29.961 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f69dc6d2f2f] 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a7c5ef2281] 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x55a7c5efefbf] 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f69db53dd90] 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr:*** Caught signal (Aborted) ** 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7f69d8dfd980 thread_name:radosgw-admin 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.963+0000 7f69d8dfd980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f69d8dfd980 time 2024-03-04T18:05:29.966607+0000 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f69dc6d2d7b] 2024-03-04T18:05:29.963 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f69dc6d2f2f] 2024-03-04T18:05:29.963 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a7c5ef2281] 2024-03-04T18:05:29.963 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x55a7c5efefbf] 2024-03-04T18:05:29.963 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:05:29.963 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f69db53dd90] 2024-03-04T18:05:29.963 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:05:29.963 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:05:29.963 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:29.964 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f69db556520] 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f69dc6d2dd5] 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f69dc6d2f2f] 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a7c5ef2281] 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x55a7c5efefbf] 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f69db53dd90] 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:29.967+0000 7f69d8dfd980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7f69d8dfd980 thread_name:radosgw-admin 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f69db556520] 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f69dc6d2dd5] 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f69dc6d2f2f] 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a7c5ef2281] 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x55a7c5efefbf] 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f69db53dd90] 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:05:29.966 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:05:29.967 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:05:29.967 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:29.967 INFO:tasks.workunit.client.0.smithi047.stderr:--- begin dump of recent events --- 2024-03-04T18:05:30.007 INFO:tasks.workunit.client.0.smithi047.stderr: -499> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.007 INFO:tasks.workunit.client.0.smithi047.stderr: -498> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -497> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -496> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -495> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -494> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -493> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -492> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -491> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -490> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -489> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -488> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.008 INFO:tasks.workunit.client.0.smithi047.stderr: -487> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -486> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -485> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command rotate-key hook 0x55a7c7cc9948 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -484> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -483> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -482> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-b con 0x55a7c7c17120 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -481> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-c con 0x55a7c7c17dc0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -480> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-a con 0x55a7c7ccff60 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -479> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -478> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -477> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -476> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:30.009 INFO:tasks.workunit.client.0.smithi047.stderr: -475> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): authenticate will time out at 2500.797424s 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -474> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): get_auth_request con 0x55a7c7c17dc0 auth_method 0 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -473> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -472> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -471> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -470> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): get_auth_request con 0x55a7c7ccff60 auth_method 0 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -469> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -468> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -467> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -466> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): get_auth_request con 0x55a7c7c17120 auth_method 0 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -465> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -464> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -463> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -462> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.010 INFO:tasks.workunit.client.0.smithi047.stderr: -461> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -460> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -459> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -458> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -457> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -456> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -455> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -454> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -453> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): handle_auth_done global_id 6762 payload 824 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -452> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient: _finish_hunting 0 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -451> 2024-03-04T18:05:29.619+0000 7f69d7116640 1 monclient: found mon.noname-a 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -450> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -449> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -448> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:30.011 INFO:tasks.workunit.client.0.smithi047.stderr: -447> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: dump: 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr: -446> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: _finish_auth 0 2024-03-04T18:05:30.012 INFO:tasks.workunit.client.0.smithi047.stderr: -445> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: _check_auth_tickets 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr: -444> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr: -443> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr: -442> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr: -441> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: dump: 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.013 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -440> 2024-03-04T18:05:29.619+0000 7f68c4e09640 4 set_mon_vals no callback set 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -439> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 monclient: authenticate success, global_id 6762 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -438> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -437> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -436> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -435> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -434> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 1 librados: init done 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -433> 2024-03-04T18:05:29.623+0000 7f68c3606640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -432> 2024-03-04T18:05:29.623+0000 7f68c3606640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -431> 2024-03-04T18:05:29.623+0000 7f68c3606640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -430> 2024-03-04T18:05:29.623+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f68bc065940 auth_method 0 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -429> 2024-03-04T18:05:29.623+0000 7f69d6114640 10 monclient: get_auth_request con 0x55a7c7cf6a70 auth_method 0 2024-03-04T18:05:30.014 INFO:tasks.workunit.client.0.smithi047.stderr: -428> 2024-03-04T18:05:29.627+0000 7f69d6915640 10 monclient: get_auth_request con 0x55a7c7b5b8e0 auth_method 0 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr: -427> 2024-03-04T18:05:29.627+0000 7f69d6114640 10 monclient: get_auth_request con 0x55a7c7d15b90 auth_method 0 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr: -426> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 10 monclient: build_initial_monmap 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr: -425> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 1 build_initial for_mkfs: 0 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr: -424> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 10 monclient: monmap: 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:05:30.015 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -423> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding auth protocol: cephx 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -422> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding auth protocol: cephx 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -421> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding auth protocol: cephx 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -420> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -419> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -418> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -417> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -416> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -415> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -414> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.016 INFO:tasks.workunit.client.0.smithi047.stderr: -413> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -412> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -411> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -410> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -409> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -408> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -407> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command objecter_requests cmddesc objecter_requests hook 0x55a7c7d1f180 EEXIST 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -406> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: init 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -405> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding auth protocol: cephx 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -404> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding auth protocol: cephx 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -403> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding auth protocol: cephx 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -402> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.017 INFO:tasks.workunit.client.0.smithi047.stderr: -401> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -400> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -399> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -398> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -397> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -396> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -395> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -394> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -393> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -392> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -391> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -390> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.018 INFO:tasks.workunit.client.0.smithi047.stderr: -389> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -388> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command rotate-key cmddesc rotate-key hook 0x55a7c7d18ba0 EEXIST 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -387> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -386> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -385> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-b con 0x55a7c7d43de0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -384> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-c con 0x55a7c7d441c0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -383> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-a con 0x55a7c7d3f380 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -382> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -381> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -380> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -379> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -378> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): authenticate will time out at 2500.808971s 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -377> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): get_auth_request con 0x55a7c7d441c0 auth_method 0 2024-03-04T18:05:30.019 INFO:tasks.workunit.client.0.smithi047.stderr: -376> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -375> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -374> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -373> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): get_auth_request con 0x55a7c7d3f380 auth_method 0 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -372> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -371> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -370> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -369> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -368> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -367> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -366> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -365> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -364> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.020 INFO:tasks.workunit.client.0.smithi047.stderr: -363> 2024-03-04T18:05:29.631+0000 7f69d6915640 10 monclient(hunting): get_auth_request con 0x55a7c7d43de0 auth_method 0 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -362> 2024-03-04T18:05:29.631+0000 7f69d6915640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -361> 2024-03-04T18:05:29.631+0000 7f69d6915640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -360> 2024-03-04T18:05:29.631+0000 7f69d6915640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -359> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): handle_auth_done global_id 6365 payload 824 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -358> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient: _finish_hunting 0 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -357> 2024-03-04T18:05:29.631+0000 7f69d6114640 1 monclient: found mon.noname-c 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -356> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -355> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -354> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr: -353> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: dump: 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.021 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr: -352> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: _finish_auth 0 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr: -351> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: _check_auth_tickets 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr: -350> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr: -349> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi047.stderr: -348> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr: -347> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: dump: 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr: -346> 2024-03-04T18:05:29.631+0000 7f69d7917640 4 set_mon_vals no callback set 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr: -345> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 monclient: authenticate success, global_id 6365 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi047.stderr: -344> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -343> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -342> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -341> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -340> 2024-03-04T18:05:29.635+0000 7f68c0e01640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -339> 2024-03-04T18:05:29.635+0000 7f68c0e01640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -338> 2024-03-04T18:05:29.635+0000 7f68c0e01640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -337> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f68b0065880 auth_method 0 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -336> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: get_monmap_and_config 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -335> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: build_initial_monmap 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -334> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 1 build_initial for_mkfs: 0 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr: -333> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: monmap: 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:05:30.024 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr: -332> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr: -331> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr: -330> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr: -329> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr: -328> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.025 INFO:tasks.workunit.client.0.smithi047.stderr: -327> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -326> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -325> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -324> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -323> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -322> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -321> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -320> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -319> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -318> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -317> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -316> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: init 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -315> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding auth protocol: cephx 2024-03-04T18:05:30.026 INFO:tasks.workunit.client.0.smithi047.stderr: -314> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding auth protocol: cephx 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -313> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding auth protocol: cephx 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -312> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -311> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -310> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -309> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -308> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -307> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -306> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -305> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -304> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -303> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -302> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -301> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.027 INFO:tasks.workunit.client.0.smithi047.stderr: -300> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -299> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -298> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command rotate-key cmddesc rotate-key hook 0x7ffc9e589548 EEXIST 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -297> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -296> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -295> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-b con 0x55a7c7d96290 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -294> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-c con 0x55a7c7d917a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -293> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-a con 0x55a7c7d9b480 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -292> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -291> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -290> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -289> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -288> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): authenticate will time out at 2500.814728s 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -287> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): get_auth_request con 0x55a7c7d917a0 auth_method 0 2024-03-04T18:05:30.028 INFO:tasks.workunit.client.0.smithi047.stderr: -286> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -285> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -284> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -283> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): get_auth_request con 0x55a7c7d9b480 auth_method 0 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -282> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -281> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -280> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -279> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -278> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -277> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -276> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -275> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -274> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.029 INFO:tasks.workunit.client.0.smithi047.stderr: -273> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient(hunting): get_auth_request con 0x55a7c7d96290 auth_method 0 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -272> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -271> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -270> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -269> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): handle_auth_done global_id 6768 payload 274 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -268> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient: _finish_hunting 0 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -267> 2024-03-04T18:05:29.635+0000 7f69d7116640 1 monclient: found mon.noname-a 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -266> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -265> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -264> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr: -263> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: dump: 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.030 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr: -262> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: _finish_auth 0 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr: -261> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: _check_auth_tickets 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr: -260> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr: -259> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr: -258> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr: -257> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: dump: 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.031 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr: -256> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 monclient: authenticate success, global_id 6768 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr: -255> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: get_monmap_and_config success 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr: -254> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 4 set_mon_vals no callback set 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr: -253> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: shutdown 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr: -252> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: build_initial_monmap 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr: -251> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 build_initial for_mkfs: 0 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr: -250> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: monmap: 2024-03-04T18:05:30.032 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr: -249> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr: -248> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.033 INFO:tasks.workunit.client.0.smithi047.stderr: -247> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -246> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -245> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -244> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -243> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -242> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -241> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -240> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -239> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -238> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -237> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -236> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -235> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -234> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.034 INFO:tasks.workunit.client.0.smithi047.stderr: -233> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: starting msgr at 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -232> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: starting objecter 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -231> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command objecter_requests cmddesc objecter_requests hook 0x55a7c7d90f30 EEXIST 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -230> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: setting wanted keys 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -229> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: calling monclient init 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -228> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: init 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -227> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding auth protocol: cephx 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -226> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding auth protocol: cephx 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -225> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding auth protocol: cephx 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -224> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -223> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -222> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -221> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -220> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.035 INFO:tasks.workunit.client.0.smithi047.stderr: -219> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -218> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -217> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -216> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -215> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -214> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -213> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -212> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -211> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -210> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command rotate-key cmddesc rotate-key hook 0x55a7c7d60c48 EEXIST 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -209> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -208> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -207> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-b con 0x55a7c7d917a0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:05:30.036 INFO:tasks.workunit.client.0.smithi047.stderr: -206> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-c con 0x55a7c7d96290 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -205> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-a con 0x55a7c7d9b480 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -204> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -203> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -202> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -201> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -200> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): authenticate will time out at 2500.818386s 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -199> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): get_auth_request con 0x55a7c7d96290 auth_method 0 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -198> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -197> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -196> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -195> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): get_auth_request con 0x55a7c7d9b480 auth_method 0 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -194> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -193> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.037 INFO:tasks.workunit.client.0.smithi047.stderr: -192> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -191> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -190> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -189> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -188> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -187> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -186> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -185> 2024-03-04T18:05:29.639+0000 7f69d6915640 10 monclient(hunting): get_auth_request con 0x55a7c7d917a0 auth_method 0 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -184> 2024-03-04T18:05:29.639+0000 7f69d6915640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -183> 2024-03-04T18:05:29.639+0000 7f69d6915640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -182> 2024-03-04T18:05:29.639+0000 7f69d6915640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -181> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): handle_auth_done global_id 6371 payload 824 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -180> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient: _finish_hunting 0 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -179> 2024-03-04T18:05:29.639+0000 7f69d6114640 1 monclient: found mon.noname-c 2024-03-04T18:05:30.038 INFO:tasks.workunit.client.0.smithi047.stderr: -178> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr: -177> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr: -176> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr: -175> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: dump: 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.039 INFO:tasks.workunit.client.0.smithi047.stderr: -174> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: _finish_auth 0 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr: -173> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: _check_auth_tickets 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr: -172> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr: -171> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr: -170> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr: -169> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: dump: 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.040 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: -168> 2024-03-04T18:05:29.639+0000 7f68ba7fc640 4 set_mon_vals no callback set 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: -167> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 monclient: authenticate success, global_id 6371 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: -166> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: -165> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: -164> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: -163> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: -162> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: init done 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: -161> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cr dump hook 0x55a7c7d86b08 2024-03-04T18:05:30.041 INFO:tasks.workunit.client.0.smithi047.stderr: -160> 2024-03-04T18:05:29.643+0000 7f68b8ff9640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -159> 2024-03-04T18:05:29.643+0000 7f68b8ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -158> 2024-03-04T18:05:29.643+0000 7f68b8ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -157> 2024-03-04T18:05:29.643+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f68a0065a00 auth_method 0 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -156> 2024-03-04T18:05:29.643+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -155> 2024-03-04T18:05:29.643+0000 7f69d7116640 10 monclient: get_auth_request con 0x55a7c7dafad0 auth_method 0 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -154> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -153> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 realm 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -152> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -151> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -150> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -149> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.042 INFO:tasks.workunit.client.0.smithi047.stderr: -148> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -147> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -146> 2024-03-04T18:05:29.647+0000 7f69d6915640 10 monclient: get_auth_request con 0x55a7c7db3d40 auth_method 0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -145> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -144> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -143> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -142> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 searching for the correct realm 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -141> 2024-03-04T18:05:29.647+0000 7f69d6114640 10 monclient: get_auth_request con 0x55a7c7db8f90 auth_method 0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -140> 2024-03-04T18:05:29.651+0000 7f69d7116640 10 monclient: get_auth_request con 0x7f69cc033020 auth_method 0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -139> 2024-03-04T18:05:29.651+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f69c80e8910 auth_method 0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -138> 2024-03-04T18:05:29.655+0000 7f69d6114640 10 monclient: get_auth_request con 0x7f69cc06a310 auth_method 0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -137> 2024-03-04T18:05:29.655+0000 7f69d6114640 0 -- 172.21.15.47:0/1258942298 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f69cc06a310 msgr2=0x7f69cc09a050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -136> 2024-03-04T18:05:29.859+0000 7f69d6114640 10 monclient: get_auth_request con 0x7f69cc06a310 auth_method 0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -135> 2024-03-04T18:05:29.863+0000 7f69d7116640 10 monclient: get_auth_request con 0x7f69c80f83f0 auth_method 0 2024-03-04T18:05:30.043 INFO:tasks.workunit.client.0.smithi047.stderr: -134> 2024-03-04T18:05:29.867+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f69d0054b30 auth_method 0 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -133> 2024-03-04T18:05:29.867+0000 7f69d7116640 0 -- 172.21.15.47:0/1258942298 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55a7c7dafad0 msgr2=0x55a7c7db1f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -132> 2024-03-04T18:05:29.867+0000 7f68b8ff9640 1 client.6371.objecter ms_handle_reset 0x55a7c7dafad0 session 0x55a7c7daf3c0 osd.6 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -131> 2024-03-04T18:05:29.867+0000 7f68b8ff9640 10 monclient: _renew_subs 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -130> 2024-03-04T18:05:29.867+0000 7f68b8ff9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -129> 2024-03-04T18:05:29.871+0000 7f69d7116640 10 monclient: get_auth_request con 0x7f68a0075d20 auth_method 0 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -128> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -127> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -126> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -125> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -124> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:30.044 INFO:tasks.workunit.client.0.smithi047.stderr: -123> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -122> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -121> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -120> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -119> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -118> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -117> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -116> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 zone default found 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -115> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 4 Realm: () 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -114> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -113> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -112> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -111> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 zonegroup default 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -110> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.045 INFO:tasks.workunit.client.0.smithi047.stderr: -109> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -108> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -107> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -106> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -105> 2024-03-04T18:05:29.887+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -104> 2024-03-04T18:05:29.887+0000 7f69d8dfd980 20 started sync module instance, tier type = 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -103> 2024-03-04T18:05:29.887+0000 7f69d8dfd980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -102> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=0 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -101> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=1 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -100> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=2 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -99> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=3 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -98> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=4 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -97> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=5 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -96> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=6 2024-03-04T18:05:30.046 INFO:tasks.workunit.client.0.smithi047.stderr: -95> 2024-03-04T18:05:29.911+0000 7f69d8dfd980 20 add_watcher() i=7 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -94> 2024-03-04T18:05:29.911+0000 7f69d8dfd980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -93> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding auth protocol: cephx 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -92> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding auth protocol: cephx 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -91> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding auth protocol: cephx 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -90> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -89> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -88> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -87> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -86> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -85> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -84> 2024-03-04T18:05:29.915+0000 7f688d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:30.047 INFO:tasks.workunit.client.0.smithi047.stderr: -83> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -82> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -81> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -80> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -79> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -78> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -77> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -76> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -75> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -74> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cache list hook 0x55a7c7d86240 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -73> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cache inspect hook 0x55a7c7d86240 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -72> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cache erase hook 0x55a7c7d86240 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -71> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cache zap hook 0x55a7c7d86240 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -70> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 note: GC not initialized 2024-03-04T18:05:30.048 INFO:tasks.workunit.client.0.smithi047.stderr: -69> 2024-03-04T18:05:29.915+0000 7f6807fff640 20 reqs_thread_entry: start 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -68> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command sync trace show hook 0x7f69c8004470 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -67> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command sync trace history hook 0x7f69c8004470 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -66> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command sync trace active hook 0x7f69c8004470 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -65> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command sync trace active_short hook 0x7f69c8004470 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -64> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -63> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 Filter name: none 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -62> 2024-03-04T18:05:29.919+0000 7f6805ffb640 20 reqs_thread_entry: start 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -61> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -60> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -59> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -58> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -57> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-03-04T18:05:30.049 INFO:tasks.workunit.client.0.smithi047.stderr: -56> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 : miss 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -55> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -54> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -53> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -52> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -51> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -50> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -49> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -48> 2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -47> 2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -46> 2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -45> 2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26, ofs0, ns] 2024-03-04T18:05:30.050 INFO:tasks.workunit.client.0.smithi047.stderr: -44> 2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595115 cookie 94179100970704 notifier 6371 bl.length()=893 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -43> 2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -42> 2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -41> 2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -40> 2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_sources= new_sources= 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -39> 2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential sources added=[] removed=[] 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -38> 2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_dests= new_dests= 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -37> 2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential dests added=[] removed=[] 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -36> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.0 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -35> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.0 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -34> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.1 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -33> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.1 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -32> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.2 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -31> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.2 2024-03-04T18:05:30.051 INFO:tasks.workunit.client.0.smithi047.stderr: -30> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.3 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -29> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.3 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -28> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.4 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -27> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.4 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -26> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.5 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -25> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.5 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -24> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.6 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -23> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.6 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -22> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.7 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -21> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.7 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -20> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.8 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -19> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.8 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -18> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.9 2024-03-04T18:05:30.052 INFO:tasks.workunit.client.0.smithi047.stderr: -17> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.9 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -16> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.10 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -15> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.10 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -14> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.11 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -13> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.11 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -12> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.0 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -11> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -10> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.2 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -9> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.3 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -8> 2024-03-04T18:05:29.947+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.4 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -7> 2024-03-04T18:05:29.947+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.5 2024-03-04T18:05:30.053 INFO:tasks.workunit.client.0.smithi047.stderr: -6> 2024-03-04T18:05:29.947+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.6 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: -5> 2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.7 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: -4> 2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.8 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: -3> 2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.9 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: -2> 2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.10 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: -1> 2024-03-04T18:05:29.963+0000 7f69d8dfd980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f69d8dfd980 time 2024-03-04T18:05:29.966607+0000 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f69dc6d2d7b] 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f69dc6d2f2f] 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a7c5ef2281] 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x55a7c5efefbf] 2024-03-04T18:05:30.054 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f69db53dd90] 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 0> 2024-03-04T18:05:29.967+0000 7f69d8dfd980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7f69d8dfd980 thread_name:radosgw-admin 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f69db556520] 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f69dc6d2dd5] 2024-03-04T18:05:30.055 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f69dc6d2f2f] 2024-03-04T18:05:30.056 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a7c5ef2281] 2024-03-04T18:05:30.056 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x55a7c5efefbf] 2024-03-04T18:05:30.056 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:05:30.056 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f69db53dd90] 2024-03-04T18:05:30.058 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr:--- logging levels --- 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 none 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 lockdep 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 context 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 crush 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_balancer 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_locker 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log 2024-03-04T18:05:30.059 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log_expire 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_migrator 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 buffer 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 timer 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 filer 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 striper 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 objecter 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rados 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_mirror 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_replay 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_pwl 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 journaler 2024-03-04T18:05:30.060 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objectcacher 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 client 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 osd 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 optracker 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objclass 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 filestore 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 journal 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 ms 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mon 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 0/10 monc 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 paxos 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 tp 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 auth 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 crypto 2024-03-04T18:05:30.061 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 finisher 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 reserver 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 heartbeatmap 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 perfcounter 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 20/20 rgw 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_sync 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_datacache 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_access 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_flight 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 javaclient 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 asok 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 throttle 2024-03-04T18:05:30.062 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 refs 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 compressor 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluestore 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluefs 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 bdev 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 kstore 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 4/ 5 rocksdb 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 fuse 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 2/ 5 mgr 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mgrc 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 dpdk 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 eventtrace 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 prioritycache 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 test 2024-03-04T18:05:30.063 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephsqlite 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_onode 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_odata 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_omap 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_tm 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_t 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_epm 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_lba 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:05:30.064 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cache 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_journal 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_device 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_backref 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 alienstore 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mclock 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cyanstore 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 ceph_exporter 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 memstore 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 trace 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: -2/-2 (syslog threshold) 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 99/99 (stderr threshold) 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 7f6805ffb640 / http_manager 2024-03-04T18:05:30.065 INFO:tasks.workunit.client.0.smithi047.stderr: 7f6807fff640 / http_manager 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f688d7e2640 / rgw_dt_lg_renew 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68b8ff9640 / ms_dispatch 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68ba7fc640 / io_context_pool 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68c0e01640 / ms_dispatch 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68c3606640 / ms_dispatch 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68c4e09640 / io_context_pool 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d6114640 / msgr-worker-2 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d6915640 / msgr-worker-1 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d7116640 / msgr-worker-0 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d7917640 / io_context_pool 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d8dfd980 / radosgw-admin 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: max_recent 500 2024-03-04T18:05:30.066 INFO:tasks.workunit.client.0.smithi047.stderr: max_new 1000 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: log_file /var/log/ceph/ceph-client.admin.94075.log 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr:--- end dump of recent events --- 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr:--- begin dump of recent events --- 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -499> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -498> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -497> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -496> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -495> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -494> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -493> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -492> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -491> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -490> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.067 INFO:tasks.workunit.client.0.smithi047.stderr: -489> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: crc 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -488> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7cc9800) adding con mode: secure 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -487> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -486> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -485> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command rotate-key hook 0x55a7c7cc9948 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -484> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -483> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -482> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-b con 0x55a7c7c17120 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -481> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-c con 0x55a7c7c17dc0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -480> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-a con 0x55a7c7ccff60 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -479> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -478> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -477> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -476> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:30.068 INFO:tasks.workunit.client.0.smithi047.stderr: -475> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient(hunting): authenticate will time out at 2500.797424s 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -474> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): get_auth_request con 0x55a7c7c17dc0 auth_method 0 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -473> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -472> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -471> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -470> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): get_auth_request con 0x55a7c7ccff60 auth_method 0 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -469> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -468> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -467> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -466> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): get_auth_request con 0x55a7c7c17120 auth_method 0 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -465> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -464> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.069 INFO:tasks.workunit.client.0.smithi047.stderr: -463> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -462> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -461> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -460> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -459> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -458> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -457> 2024-03-04T18:05:29.619+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -456> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -455> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -454> 2024-03-04T18:05:29.619+0000 7f69d6915640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -453> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient(hunting): handle_auth_done global_id 6762 payload 824 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -452> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient: _finish_hunting 0 2024-03-04T18:05:30.070 INFO:tasks.workunit.client.0.smithi047.stderr: -451> 2024-03-04T18:05:29.619+0000 7f69d7116640 1 monclient: found mon.noname-a 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr: -450> 2024-03-04T18:05:29.619+0000 7f69d7116640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr: -449> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr: -448> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr: -447> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: dump: 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.071 INFO:tasks.workunit.client.0.smithi047.stderr: -446> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: _finish_auth 0 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr: -445> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: _check_auth_tickets 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr: -444> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr: -443> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr: -442> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr: -441> 2024-03-04T18:05:29.619+0000 7f68c3606640 10 monclient: dump: 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.072 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -440> 2024-03-04T18:05:29.619+0000 7f68c4e09640 4 set_mon_vals no callback set 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -439> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 5 monclient: authenticate success, global_id 6762 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -438> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -437> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -436> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -435> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -434> 2024-03-04T18:05:29.619+0000 7f69d8dfd980 1 librados: init done 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -433> 2024-03-04T18:05:29.623+0000 7f68c3606640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -432> 2024-03-04T18:05:29.623+0000 7f68c3606640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -431> 2024-03-04T18:05:29.623+0000 7f68c3606640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.073 INFO:tasks.workunit.client.0.smithi047.stderr: -430> 2024-03-04T18:05:29.623+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f68bc065940 auth_method 0 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr: -429> 2024-03-04T18:05:29.623+0000 7f69d6114640 10 monclient: get_auth_request con 0x55a7c7cf6a70 auth_method 0 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr: -428> 2024-03-04T18:05:29.627+0000 7f69d6915640 10 monclient: get_auth_request con 0x55a7c7b5b8e0 auth_method 0 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr: -427> 2024-03-04T18:05:29.627+0000 7f69d6114640 10 monclient: get_auth_request con 0x55a7c7d15b90 auth_method 0 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr: -426> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 10 monclient: build_initial_monmap 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr: -425> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 1 build_initial for_mkfs: 0 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr: -424> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 10 monclient: monmap: 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.074 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -423> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding auth protocol: cephx 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -422> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding auth protocol: cephx 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -421> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding auth protocol: cephx 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -420> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -419> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -418> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -417> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -416> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -415> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -414> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.075 INFO:tasks.workunit.client.0.smithi047.stderr: -413> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -412> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -411> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -410> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: crc 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -409> 2024-03-04T18:05:29.627+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d19a70) adding con mode: secure 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -408> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -407> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command objecter_requests cmddesc objecter_requests hook 0x55a7c7d1f180 EEXIST 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -406> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: init 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -405> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding auth protocol: cephx 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -404> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding auth protocol: cephx 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -403> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding auth protocol: cephx 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -402> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.076 INFO:tasks.workunit.client.0.smithi047.stderr: -401> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -400> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -399> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -398> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -397> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -396> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -395> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -394> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -393> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -392> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: crc 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -391> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d18a58) adding con mode: secure 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -390> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -389> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -388> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command rotate-key cmddesc rotate-key hook 0x55a7c7d18ba0 EEXIST 2024-03-04T18:05:30.077 INFO:tasks.workunit.client.0.smithi047.stderr: -387> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -386> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -385> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-b con 0x55a7c7d43de0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -384> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-c con 0x55a7c7d441c0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -383> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-a con 0x55a7c7d3f380 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -382> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -381> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -380> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -379> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -378> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient(hunting): authenticate will time out at 2500.808971s 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -377> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): get_auth_request con 0x55a7c7d441c0 auth_method 0 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -376> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -375> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -374> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -373> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): get_auth_request con 0x55a7c7d3f380 auth_method 0 2024-03-04T18:05:30.078 INFO:tasks.workunit.client.0.smithi047.stderr: -372> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -371> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -370> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -369> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -368> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -367> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -366> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -365> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -364> 2024-03-04T18:05:29.631+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -363> 2024-03-04T18:05:29.631+0000 7f69d6915640 10 monclient(hunting): get_auth_request con 0x55a7c7d43de0 auth_method 0 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -362> 2024-03-04T18:05:29.631+0000 7f69d6915640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -361> 2024-03-04T18:05:29.631+0000 7f69d6915640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -360> 2024-03-04T18:05:29.631+0000 7f69d6915640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -359> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient(hunting): handle_auth_done global_id 6365 payload 824 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -358> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient: _finish_hunting 0 2024-03-04T18:05:30.079 INFO:tasks.workunit.client.0.smithi047.stderr: -357> 2024-03-04T18:05:29.631+0000 7f69d6114640 1 monclient: found mon.noname-c 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr: -356> 2024-03-04T18:05:29.631+0000 7f69d6114640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr: -355> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr: -354> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr: -353> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: dump: 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.080 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr: -352> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: _finish_auth 0 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr: -351> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: _check_auth_tickets 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr: -350> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr: -349> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr: -348> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr: -347> 2024-03-04T18:05:29.631+0000 7f68c0e01640 10 monclient: dump: 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.081 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -346> 2024-03-04T18:05:29.631+0000 7f69d7917640 4 set_mon_vals no callback set 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -345> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 5 monclient: authenticate success, global_id 6365 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -344> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -343> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -342> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -341> 2024-03-04T18:05:29.631+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -340> 2024-03-04T18:05:29.635+0000 7f68c0e01640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -339> 2024-03-04T18:05:29.635+0000 7f68c0e01640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -338> 2024-03-04T18:05:29.635+0000 7f68c0e01640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -337> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f68b0065880 auth_method 0 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -336> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: get_monmap_and_config 2024-03-04T18:05:30.082 INFO:tasks.workunit.client.0.smithi047.stderr: -335> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: build_initial_monmap 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr: -334> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 1 build_initial for_mkfs: 0 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr: -333> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: monmap: 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr: -332> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.083 INFO:tasks.workunit.client.0.smithi047.stderr: -331> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -330> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -329> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -328> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -327> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -326> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -325> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -324> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -323> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -322> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -321> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.084 INFO:tasks.workunit.client.0.smithi047.stderr: -320> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -319> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -318> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -317> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -316> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: init 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -315> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding auth protocol: cephx 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -314> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding auth protocol: cephx 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -313> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding auth protocol: cephx 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -312> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -311> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -310> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -309> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -308> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -307> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.085 INFO:tasks.workunit.client.0.smithi047.stderr: -306> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -305> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -304> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -303> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -302> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: crc 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -301> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589400) adding con mode: secure 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -300> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -299> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -298> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command rotate-key cmddesc rotate-key hook 0x7ffc9e589548 EEXIST 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -297> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -296> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -295> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-b con 0x55a7c7d96290 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -294> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-c con 0x55a7c7d917a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -293> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-a con 0x55a7c7d9b480 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi047.stderr: -292> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -291> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -290> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -289> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -288> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient(hunting): authenticate will time out at 2500.814728s 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -287> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): get_auth_request con 0x55a7c7d917a0 auth_method 0 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -286> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -285> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -284> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -283> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): get_auth_request con 0x55a7c7d9b480 auth_method 0 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -282> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -281> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -280> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.087 INFO:tasks.workunit.client.0.smithi047.stderr: -279> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -278> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -277> 2024-03-04T18:05:29.635+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -276> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -275> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -274> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -273> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient(hunting): get_auth_request con 0x55a7c7d96290 auth_method 0 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -272> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -271> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -270> 2024-03-04T18:05:29.635+0000 7f69d6915640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -269> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient(hunting): handle_auth_done global_id 6768 payload 274 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -268> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient: _finish_hunting 0 2024-03-04T18:05:30.088 INFO:tasks.workunit.client.0.smithi047.stderr: -267> 2024-03-04T18:05:29.635+0000 7f69d7116640 1 monclient: found mon.noname-a 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr: -266> 2024-03-04T18:05:29.635+0000 7f69d7116640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr: -265> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr: -264> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr: -263> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: dump: 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.089 INFO:tasks.workunit.client.0.smithi047.stderr: -262> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: _finish_auth 0 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr: -261> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: _check_auth_tickets 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr: -260> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr: -259> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr: -258> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr: -257> 2024-03-04T18:05:29.635+0000 7f68ba7fc640 10 monclient: dump: 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.090 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr: -256> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 5 monclient: authenticate success, global_id 6768 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr: -255> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 10 monclient: get_monmap_and_config success 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr: -254> 2024-03-04T18:05:29.635+0000 7f69d8dfd980 4 set_mon_vals no callback set 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr: -253> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: shutdown 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr: -252> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: build_initial_monmap 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr: -251> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 build_initial for_mkfs: 0 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr: -250> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: monmap: 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:05:30.091 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr: -249> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr: -248> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr: -247> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding auth protocol: cephx 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr: -246> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr: -245> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr: -244> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr: -243> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.092 INFO:tasks.workunit.client.0.smithi047.stderr: -242> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -241> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -240> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -239> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -238> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -237> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -236> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: crc 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -235> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d61930) adding con mode: secure 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -234> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -233> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: starting msgr at 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -232> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: starting objecter 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -231> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command objecter_requests cmddesc objecter_requests hook 0x55a7c7d90f30 EEXIST 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -230> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: setting wanted keys 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -229> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: calling monclient init 2024-03-04T18:05:30.093 INFO:tasks.workunit.client.0.smithi047.stderr: -228> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: init 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -227> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding auth protocol: cephx 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -226> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding auth protocol: cephx 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -225> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding auth protocol: cephx 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -224> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -223> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -222> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -221> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -220> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -219> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -218> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -217> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -216> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -215> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.094 INFO:tasks.workunit.client.0.smithi047.stderr: -214> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: crc 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -213> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 AuthRegistry(0x55a7c7d60b00) adding con mode: secure 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -212> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -211> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -210> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command rotate-key cmddesc rotate-key hook 0x55a7c7d60c48 EEXIST 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -209> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -208> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -207> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-b con 0x55a7c7d917a0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -206> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-c con 0x55a7c7d96290 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -205> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): picked mon.noname-a con 0x55a7c7d9b480 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -204> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -203> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -202> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -201> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:30.095 INFO:tasks.workunit.client.0.smithi047.stderr: -200> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient(hunting): authenticate will time out at 2500.818386s 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -199> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): get_auth_request con 0x55a7c7d96290 auth_method 0 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -198> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -197> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -196> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -195> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): get_auth_request con 0x55a7c7d9b480 auth_method 0 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -194> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -193> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -192> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -191> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -190> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -189> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -188> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:30.096 INFO:tasks.workunit.client.0.smithi047.stderr: -187> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -186> 2024-03-04T18:05:29.639+0000 7f69d7116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -185> 2024-03-04T18:05:29.639+0000 7f69d6915640 10 monclient(hunting): get_auth_request con 0x55a7c7d917a0 auth_method 0 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -184> 2024-03-04T18:05:29.639+0000 7f69d6915640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -183> 2024-03-04T18:05:29.639+0000 7f69d6915640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -182> 2024-03-04T18:05:29.639+0000 7f69d6915640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -181> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient(hunting): handle_auth_done global_id 6371 payload 824 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -180> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient: _finish_hunting 0 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -179> 2024-03-04T18:05:29.639+0000 7f69d6114640 1 monclient: found mon.noname-c 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -178> 2024-03-04T18:05:29.639+0000 7f69d6114640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -177> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -176> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr: -175> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: dump: 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.097 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr: -174> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: _finish_auth 0 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr: -173> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: _check_auth_tickets 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr: -172> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:30.098 INFO:tasks.workunit.client.0.smithi047.stderr: -171> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr: -170> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr: -169> 2024-03-04T18:05:29.639+0000 7f68b8ff9640 10 monclient: dump: 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr: -168> 2024-03-04T18:05:29.639+0000 7f68ba7fc640 4 set_mon_vals no callback set 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr: -167> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 monclient: authenticate success, global_id 6371 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr: -166> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.099 INFO:tasks.workunit.client.0.smithi047.stderr: -165> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -164> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _renew_subs 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -163> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -162> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 1 librados: init done 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -161> 2024-03-04T18:05:29.639+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cr dump hook 0x55a7c7d86b08 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -160> 2024-03-04T18:05:29.643+0000 7f68b8ff9640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -159> 2024-03-04T18:05:29.643+0000 7f68b8ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -158> 2024-03-04T18:05:29.643+0000 7f68b8ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -157> 2024-03-04T18:05:29.643+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f68a0065a00 auth_method 0 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -156> 2024-03-04T18:05:29.643+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -155> 2024-03-04T18:05:29.643+0000 7f69d7116640 10 monclient: get_auth_request con 0x55a7c7dafad0 auth_method 0 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -154> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.100 INFO:tasks.workunit.client.0.smithi047.stderr: -153> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 realm 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -152> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -151> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -150> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -149> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -148> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -147> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -146> 2024-03-04T18:05:29.647+0000 7f69d6915640 10 monclient: get_auth_request con 0x55a7c7db3d40 auth_method 0 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -145> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -144> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -143> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -142> 2024-03-04T18:05:29.647+0000 7f69d8dfd980 20 searching for the correct realm 2024-03-04T18:05:30.101 INFO:tasks.workunit.client.0.smithi047.stderr: -141> 2024-03-04T18:05:29.647+0000 7f69d6114640 10 monclient: get_auth_request con 0x55a7c7db8f90 auth_method 0 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -140> 2024-03-04T18:05:29.651+0000 7f69d7116640 10 monclient: get_auth_request con 0x7f69cc033020 auth_method 0 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -139> 2024-03-04T18:05:29.651+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f69c80e8910 auth_method 0 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -138> 2024-03-04T18:05:29.655+0000 7f69d6114640 10 monclient: get_auth_request con 0x7f69cc06a310 auth_method 0 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -137> 2024-03-04T18:05:29.655+0000 7f69d6114640 0 -- 172.21.15.47:0/1258942298 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f69cc06a310 msgr2=0x7f69cc09a050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -136> 2024-03-04T18:05:29.859+0000 7f69d6114640 10 monclient: get_auth_request con 0x7f69cc06a310 auth_method 0 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -135> 2024-03-04T18:05:29.863+0000 7f69d7116640 10 monclient: get_auth_request con 0x7f69c80f83f0 auth_method 0 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -134> 2024-03-04T18:05:29.867+0000 7f69d6915640 10 monclient: get_auth_request con 0x7f69d0054b30 auth_method 0 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -133> 2024-03-04T18:05:29.867+0000 7f69d7116640 0 -- 172.21.15.47:0/1258942298 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55a7c7dafad0 msgr2=0x55a7c7db1f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -132> 2024-03-04T18:05:29.867+0000 7f68b8ff9640 1 client.6371.objecter ms_handle_reset 0x55a7c7dafad0 session 0x55a7c7daf3c0 osd.6 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -131> 2024-03-04T18:05:29.867+0000 7f68b8ff9640 10 monclient: _renew_subs 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -130> 2024-03-04T18:05:29.867+0000 7f68b8ff9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -129> 2024-03-04T18:05:29.871+0000 7f69d7116640 10 monclient: get_auth_request con 0x7f68a0075d20 auth_method 0 2024-03-04T18:05:30.102 INFO:tasks.workunit.client.0.smithi047.stderr: -128> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -127> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -126> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -125> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -124> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -123> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -122> 2024-03-04T18:05:29.879+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -121> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -120> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -119> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -118> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -117> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -116> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 zone default found 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -115> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 4 Realm: () 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -114> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:30.103 INFO:tasks.workunit.client.0.smithi047.stderr: -113> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -112> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -111> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 zonegroup default 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -110> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -109> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -108> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -107> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -106> 2024-03-04T18:05:29.883+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -105> 2024-03-04T18:05:29.887+0000 7f69d8dfd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -104> 2024-03-04T18:05:29.887+0000 7f69d8dfd980 20 started sync module instance, tier type = 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -103> 2024-03-04T18:05:29.887+0000 7f69d8dfd980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -102> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=0 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -101> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=1 2024-03-04T18:05:30.104 INFO:tasks.workunit.client.0.smithi047.stderr: -100> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=2 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -99> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=3 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -98> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=4 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -97> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=5 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -96> 2024-03-04T18:05:29.907+0000 7f69d8dfd980 20 add_watcher() i=6 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -95> 2024-03-04T18:05:29.911+0000 7f69d8dfd980 20 add_watcher() i=7 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -94> 2024-03-04T18:05:29.911+0000 7f69d8dfd980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -93> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding auth protocol: cephx 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -92> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding auth protocol: cephx 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -91> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding auth protocol: cephx 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -90> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -89> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -88> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -87> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -86> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.105 INFO:tasks.workunit.client.0.smithi047.stderr: -85> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -84> 2024-03-04T18:05:29.915+0000 7f688d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -83> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -82> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -81> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -80> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -79> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: crc 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -78> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 AuthRegistry(0x7ffc9e589890) adding con mode: secure 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -77> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -76> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -75> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -74> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cache list hook 0x55a7c7d86240 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -73> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cache inspect hook 0x55a7c7d86240 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -72> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cache erase hook 0x55a7c7d86240 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -71> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command cache zap hook 0x55a7c7d86240 2024-03-04T18:05:30.106 INFO:tasks.workunit.client.0.smithi047.stderr: -70> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 note: GC not initialized 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -69> 2024-03-04T18:05:29.915+0000 7f6807fff640 20 reqs_thread_entry: start 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -68> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command sync trace show hook 0x7f69c8004470 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -67> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command sync trace history hook 0x7f69c8004470 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -66> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command sync trace active hook 0x7f69c8004470 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -65> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 5 asok(0x55a7c7ae2130) register_command sync trace active_short hook 0x7f69c8004470 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -64> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -63> 2024-03-04T18:05:29.915+0000 7f69d8dfd980 20 Filter name: none 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -62> 2024-03-04T18:05:29.919+0000 7f6805ffb640 20 reqs_thread_entry: start 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -61> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -60> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -59> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -58> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -57> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -56> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 : miss 2024-03-04T18:05:30.107 INFO:tasks.workunit.client.0.smithi047.stderr: -55> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -54> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -53> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -52> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -51> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -50> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -49> 2024-03-04T18:05:29.919+0000 7f69d8dfd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -48> 2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -47> 2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -46> 2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -45> 2024-03-04T18:05:29.931+0000 7f69d8dfd980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26, ofs0, ns] 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -44> 2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595115 cookie 94179100970704 notifier 6371 bl.length()=893 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -43> 2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:05:30.108 INFO:tasks.workunit.client.0.smithi047.stderr: -42> 2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -41> 2024-03-04T18:05:29.931+0000 7f68ba7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -40> 2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_sources= new_sources= 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -39> 2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential sources added=[] removed=[] 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -38> 2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_dests= new_dests= 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -37> 2024-03-04T18:05:29.935+0000 7f69d8dfd980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential dests added=[] removed=[] 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -36> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.0 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -35> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.0 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -34> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.1 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -33> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.1 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -32> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.2 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -31> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.2 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -30> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.3 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -29> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.3 2024-03-04T18:05:30.109 INFO:tasks.workunit.client.0.smithi047.stderr: -28> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.4 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -27> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.4 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -26> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.5 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -25> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.5 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -24> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.6 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -23> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.6 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -22> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.7 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -21> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.7 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -20> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.8 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -19> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.8 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -18> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.9 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -17> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.9 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -16> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.10 2024-03-04T18:05:30.110 INFO:tasks.workunit.client.0.smithi047.stderr: -15> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.10 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -14> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.11 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -13> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.11 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -12> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.0 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -11> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -10> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.2 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -9> 2024-03-04T18:05:29.943+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.3 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -8> 2024-03-04T18:05:29.947+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.4 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -7> 2024-03-04T18:05:29.947+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.5 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -6> 2024-03-04T18:05:29.947+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.6 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -5> 2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.7 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -4> 2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.8 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -3> 2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.9 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -2> 2024-03-04T18:05:29.951+0000 7f69d8dfd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.10 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr: -1> 2024-03-04T18:05:29.963+0000 7f69d8dfd980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f69d8dfd980 time 2024-03-04T18:05:29.966607+0000 2024-03-04T18:05:30.111 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f69dc6d2d7b] 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f69dc6d2f2f] 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a7c5ef2281] 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x55a7c5efefbf] 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f69db53dd90] 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.112 INFO:tasks.workunit.client.0.smithi047.stderr: 0> 2024-03-04T18:05:29.967+0000 7f69d8dfd980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7f69d8dfd980 thread_name:radosgw-admin 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f69db556520] 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f69dc6d2dd5] 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f69dc6d2f2f] 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a7c5ef2281] 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x55a7c5efefbf] 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:05:30.113 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f69db53dd90] 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr:--- logging levels --- 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 none 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 lockdep 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 context 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 crush 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_balancer 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_locker 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log_expire 2024-03-04T18:05:30.114 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_migrator 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 buffer 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 timer 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 filer 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 striper 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 objecter 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rados 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_mirror 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_replay 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_pwl 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 journaler 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objectcacher 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:05:30.115 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 client 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 osd 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 optracker 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objclass 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 filestore 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 journal 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 ms 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mon 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 0/10 monc 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 paxos 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 tp 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 auth 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 crypto 2024-03-04T18:05:30.116 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 finisher 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 reserver 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 heartbeatmap 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 perfcounter 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 20/20 rgw 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_sync 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_datacache 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_access 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_flight 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 javaclient 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 asok 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 throttle 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 refs 2024-03-04T18:05:30.117 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 compressor 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluestore 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluefs 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 bdev 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 kstore 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 4/ 5 rocksdb 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 fuse 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 2/ 5 mgr 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mgrc 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 dpdk 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 eventtrace 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 prioritycache 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 test 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:05:30.118 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephsqlite 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_onode 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_odata 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_omap 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_tm 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_t 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_epm 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_lba 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cache 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_journal 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_device 2024-03-04T18:05:30.119 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_backref 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 alienstore 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mclock 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cyanstore 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 ceph_exporter 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 memstore 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 trace 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: -2/-2 (syslog threshold) 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 99/99 (stderr threshold) 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 7f6805ffb640 / http_manager 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 7f6807fff640 / http_manager 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 7f688d7e2640 / rgw_dt_lg_renew 2024-03-04T18:05:30.120 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68b8ff9640 / ms_dispatch 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68ba7fc640 / io_context_pool 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68c0e01640 / ms_dispatch 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68c3606640 / ms_dispatch 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: 7f68c4e09640 / io_context_pool 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d6114640 / msgr-worker-2 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d6915640 / msgr-worker-1 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d7116640 / msgr-worker-0 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d7917640 / io_context_pool 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: 7f69d8dfd980 / radosgw-admin 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: max_recent 500 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: max_new 1000 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr: log_file /var/lib/ceph/crash/2024-03-04T18:05:29.971955Z_a64509e9-46bb-4ffe-b5e5-b10c8fc84d8f/log 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr:--- end dump of recent events --- 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr:Aborted 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-03-04T18:05:30.122 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:30.122 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.051+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.051+0000 7fe948128980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.051+0000 7fe948128980 20 realm 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.051+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.051+0000 7fe948128980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.051+0000 7fe948128980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.051+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.051+0000 7fe948128980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.051+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.055+0000 7fe948128980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.055+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.055+0000 7fe948128980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.055+0000 7fe948128980 20 searching for the correct realm 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.063+0000 7fe945c40640 0 -- 172.21.15.47:0/4060906582 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fe938060030 msgr2=0x7fe938059630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.067+0000 7fe945c40640 0 -- 172.21.15.47:0/4060906582 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fe9400ab070 msgr2=0x7fe9400b7990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 zone default found 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 4 Realm: () 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 zonegroup default 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.075+0000 7fe948128980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.079+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.079+0000 7fe948128980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.079+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.079+0000 7fe948128980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.079+0000 7fe948128980 20 started sync module instance, tier type = 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.079+0000 7fe948128980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.095+0000 7fe948128980 20 add_watcher() i=0 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.095+0000 7fe948128980 20 add_watcher() i=1 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.095+0000 7fe948128980 20 add_watcher() i=2 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.095+0000 7fe948128980 20 add_watcher() i=3 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.095+0000 7fe948128980 20 add_watcher() i=4 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.095+0000 7fe948128980 20 add_watcher() i=5 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.095+0000 7fe948128980 20 add_watcher() i=6 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.099+0000 7fe948128980 20 add_watcher() i=7 2024-03-04T18:05:30.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.099+0000 7fe948128980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe7f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 5 note: GC not initialized 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe7f67fc640 20 reqs_thread_entry: start 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 init_complete bucket index max shards: 11 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 Filter name: none 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe767fff640 20 reqs_thread_entry: start 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 rados_obj.operate() r=0 bl.length=413 2024-03-04T18:05:30.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.103+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.107+0000 7fe948128980 20 rados_obj.operate() r=0 bl.length=413 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.107+0000 7fe948128980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.107+0000 7fe948128980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.107+0000 7fe948128980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.107+0000 7fe7f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.115+0000 7fe948128980 20 remove_watcher() i=0 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.115+0000 7fe948128980 2 removed watcher, disabling cache 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.115+0000 7fe948128980 20 remove_watcher() i=1 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.119+0000 7fe948128980 20 remove_watcher() i=2 2024-03-04T18:05:30.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.119+0000 7fe948128980 20 remove_watcher() i=3 2024-03-04T18:05:30.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.123+0000 7fe948128980 20 remove_watcher() i=4 2024-03-04T18:05:30.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.123+0000 7fe948128980 20 remove_watcher() i=5 2024-03-04T18:05:30.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.127+0000 7fe948128980 20 remove_watcher() i=6 2024-03-04T18:05:30.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:05:30.127+0000 7fe948128980 20 remove_watcher() i=7 2024-03-04T18:05:30.142 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.142 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:05:30.142 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-commit-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.26 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:05:30.142 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:05:30.143 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:30.143 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:00.419 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:00.419 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:00.502 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:06:00.540 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:00.540 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:00.584 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.586+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 20 realm 2024-03-04T18:06:00.586 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:00.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:00.588 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.590+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.594+0000 7f30bc53e980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:00.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.594+0000 7f30bc53e980 20 searching for the correct realm 2024-03-04T18:06:00.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.602+0000 7f30ba857640 0 -- 172.21.15.47:0/812496119 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f30b00b8950 msgr2=0x7f30b00ce410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:00.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.606+0000 7f30ba056640 0 -- 172.21.15.47:0/812496119 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f30b4015030 msgr2=0x7f30b4076110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:00.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:00.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:00.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:00.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:00.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:00.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:00.610 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.611 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 zone default found 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 4 Realm: () 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 zonegroup default 2024-03-04T18:06:00.612 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.614+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.618+0000 7f30bc53e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.618+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.613 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.618+0000 7f30bc53e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.618+0000 7f30bc53e980 20 started sync module instance, tier type = 2024-03-04T18:06:00.614 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.618+0000 7f30bc53e980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:00.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.634+0000 7f30bc53e980 20 add_watcher() i=0 2024-03-04T18:06:00.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.634+0000 7f30bc53e980 20 add_watcher() i=1 2024-03-04T18:06:00.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.634+0000 7f30bc53e980 20 add_watcher() i=2 2024-03-04T18:06:00.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.634+0000 7f30bc53e980 20 add_watcher() i=3 2024-03-04T18:06:00.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.634+0000 7f30bc53e980 20 add_watcher() i=4 2024-03-04T18:06:00.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.634+0000 7f30bc53e980 20 add_watcher() i=5 2024-03-04T18:06:00.632 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.634+0000 7f30bc53e980 20 add_watcher() i=6 2024-03-04T18:06:00.634 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.638+0000 7f30bc53e980 20 add_watcher() i=7 2024-03-04T18:06:00.635 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.638+0000 7f30bc53e980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:00.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f2f70fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:00.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f30bc53e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:00.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f30bc53e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:00.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f30bc53e980 5 note: GC not initialized 2024-03-04T18:06:00.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f2eeb7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:00.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f30bc53e980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:00.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f30bc53e980 20 Filter name: none 2024-03-04T18:06:00.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f2ee97fa640 20 reqs_thread_entry: start 2024-03-04T18:06:00.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f30bc53e980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-03-04T18:06:00.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.642+0000 7f30bc53e980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 : miss 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:00.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.646+0000 7f30bc53e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 2024-03-04T18:06:00.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f30bc53e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:06:00.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f30bc53e980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:06:00.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f30bc53e980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:00.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f30bc53e980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26, ofs0, ns] 2024-03-04T18:06:00.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f2f9d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595117 cookie 94365458601168 notifier 6407 bl.length()=893 2024-03-04T18:06:00.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f2f9d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:06:00.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f2f9d7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:06:00.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f2f9d7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:00.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f30bc53e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_sources= new_sources= 2024-03-04T18:06:00.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f30bc53e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential sources added=[] removed=[] 2024-03-04T18:06:00.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f30bc53e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_dests= new_dests= 2024-03-04T18:06:00.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.658+0000 7f30bc53e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential dests added=[] removed=[] 2024-03-04T18:06:00.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.0 2024-03-04T18:06:00.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.0 2024-03-04T18:06:00.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.1 2024-03-04T18:06:00.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.1 2024-03-04T18:06:00.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.2 2024-03-04T18:06:00.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.2 2024-03-04T18:06:00.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.3 2024-03-04T18:06:00.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.3 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.4 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.4 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.5 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.5 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.6 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.6 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.7 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.7 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.8 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.8 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.9 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.9 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.10 2024-03-04T18:06:00.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.10 2024-03-04T18:06:00.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.11 2024-03-04T18:06:00.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1.11 2024-03-04T18:06:00.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.0 2024-03-04T18:06:00.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.1 2024-03-04T18:06:00.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.666+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.2 2024-03-04T18:06:00.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.670+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.3 2024-03-04T18:06:00.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.670+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.4 2024-03-04T18:06:00.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.670+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.5 2024-03-04T18:06:00.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.670+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.6 2024-03-04T18:06:00.669 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.674+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.7 2024-03-04T18:06:00.669 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.674+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.8 2024-03-04T18:06:00.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.674+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.9 2024-03-04T18:06:00.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.674+0000 7f30bc53e980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.26.10 2024-03-04T18:06:00.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.682+0000 7f30bc53e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:00.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.682+0000 7f30bc53e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 2024-03-04T18:06:00.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.686+0000 7f30bc53e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:06:00.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.686+0000 7f30bc53e980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:06:00.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.686+0000 7f30bc53e980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:00.682 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.686+0000 7f30bc53e980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26, ofs0, ns] 2024-03-04T18:06:00.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.686+0000 7f2f9dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595118 cookie 94365458601168 notifier 6407 bl.length()=861 2024-03-04T18:06:00.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.686+0000 7f2f9dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 info.flags=0x17 2024-03-04T18:06:00.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.686+0000 7f2f9dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:797b67a4-b876-4d04-b122-10afd858d465.4453.26 to cache LRU end 2024-03-04T18:06:00.683 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.686+0000 7f2f9dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:00.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.690+0000 7f30bc53e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_sources= new_sources= 2024-03-04T18:06:00.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.690+0000 7f30bc53e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential sources added=[] removed=[] 2024-03-04T18:06:00.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.690+0000 7f30bc53e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): orig_dests= new_dests= 2024-03-04T18:06:00.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.690+0000 7f30bc53e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[797b67a4-b876-4d04-b122-10afd858d465.4453.26]): potential dests added=[] removed=[] 2024-03-04T18:06:00.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.694+0000 7f30bc53e980 1 execute INFO: reshard of bucket "abort-at-commit-target-layout" completed successfully 2024-03-04T18:06:00.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.698+0000 7f2f70fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:00.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.702+0000 7f30bc53e980 20 remove_watcher() i=0 2024-03-04T18:06:00.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.702+0000 7f30bc53e980 2 removed watcher, disabling cache 2024-03-04T18:06:00.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.706+0000 7f30bc53e980 20 remove_watcher() i=1 2024-03-04T18:06:00.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.706+0000 7f30bc53e980 20 remove_watcher() i=2 2024-03-04T18:06:00.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.710+0000 7f30bc53e980 20 remove_watcher() i=3 2024-03-04T18:06:00.708 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.714+0000 7f30bc53e980 20 remove_watcher() i=4 2024-03-04T18:06:00.710 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.714+0000 7f30bc53e980 20 remove_watcher() i=5 2024-03-04T18:06:00.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.718+0000 7f30bc53e980 20 remove_watcher() i=6 2024-03-04T18:06:00.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.722+0000 7f30bc53e980 20 remove_watcher() i=7 2024-03-04T18:06:00.732 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-03-04T18:06:00.761 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:00.761 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:00.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.810+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.810+0000 7fc6ecadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.810+0000 7fc6ecadf980 20 realm 2024-03-04T18:06:00.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.810+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.810+0000 7fc6ecadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.810+0000 7fc6ecadf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:00.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.810+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.810+0000 7fc6ecadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.810+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.808 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.814+0000 7fc6ecadf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:00.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.814+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.814+0000 7fc6ecadf980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:00.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.814+0000 7fc6ecadf980 20 searching for the correct realm 2024-03-04T18:06:00.816 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.818+0000 7fc6e3fff640 0 -- 172.21.15.47:0/3828280909 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fc6d40b9bf0 msgr2=0x7fc6d40fe120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:00.822 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.826+0000 7fc6eadf8640 0 -- 172.21.15.47:0/3828280909 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55704d76e710 msgr2=0x55704d770b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:00.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.834+0000 7fc6ecadf980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:00.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.834+0000 7fc6ecadf980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:00.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.834+0000 7fc6ecadf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:00.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.834+0000 7fc6ecadf980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:00.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.834+0000 7fc6ecadf980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:00.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.834+0000 7fc6ecadf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:00.832 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.834+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.834+0000 7fc6ecadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.834+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 zone default found 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 4 Realm: () 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 zonegroup default 2024-03-04T18:06:00.834 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 started sync module instance, tier type = 2024-03-04T18:06:00.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.838+0000 7fc6ecadf980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:00.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.858+0000 7fc6ecadf980 20 add_watcher() i=0 2024-03-04T18:06:00.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.858+0000 7fc6ecadf980 20 add_watcher() i=1 2024-03-04T18:06:00.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.858+0000 7fc6ecadf980 20 add_watcher() i=2 2024-03-04T18:06:00.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.858+0000 7fc6ecadf980 20 add_watcher() i=3 2024-03-04T18:06:00.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.858+0000 7fc6ecadf980 20 add_watcher() i=4 2024-03-04T18:06:00.853 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.858+0000 7fc6ecadf980 20 add_watcher() i=5 2024-03-04T18:06:00.854 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.858+0000 7fc6ecadf980 20 add_watcher() i=6 2024-03-04T18:06:00.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.858+0000 7fc6ecadf980 20 add_watcher() i=7 2024-03-04T18:06:00.856 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.858+0000 7fc6ecadf980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:00.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.862+0000 7fc5a0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:00.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.862+0000 7fc6ecadf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:00.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.862+0000 7fc6ecadf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:00.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.862+0000 7fc6ecadf980 5 note: GC not initialized 2024-03-04T18:06:00.860 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.862+0000 7fc51b7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:00.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.862+0000 7fc6ecadf980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:00.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.862+0000 7fc6ecadf980 20 Filter name: none 2024-03-04T18:06:00.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.862+0000 7fc5197fa640 20 reqs_thread_entry: start 2024-03-04T18:06:00.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.861 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:00.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:06:00.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:00.862 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:00.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 rados_obj.operate() r=0 bl.length=381 2024-03-04T18:06:00.863 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.866+0000 7fc6ecadf980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:00.865 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.870+0000 7fc6ecadf980 20 rados->read ofs=0 len=0 2024-03-04T18:06:00.866 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.870+0000 7fc6ecadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:00.867 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.870+0000 7fc5a0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:00.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.874+0000 7fc6ecadf980 20 remove_watcher() i=0 2024-03-04T18:06:00.871 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.874+0000 7fc6ecadf980 2 removed watcher, disabling cache 2024-03-04T18:06:00.873 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.878+0000 7fc6ecadf980 20 remove_watcher() i=1 2024-03-04T18:06:00.875 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.878+0000 7fc6ecadf980 20 remove_watcher() i=2 2024-03-04T18:06:00.877 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.882+0000 7fc6ecadf980 20 remove_watcher() i=3 2024-03-04T18:06:00.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.882+0000 7fc6ecadf980 20 remove_watcher() i=4 2024-03-04T18:06:00.881 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.886+0000 7fc6ecadf980 20 remove_watcher() i=5 2024-03-04T18:06:00.884 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.886+0000 7fc6ecadf980 20 remove_watcher() i=6 2024-03-04T18:06:00.887 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:00.890+0000 7fc6ecadf980 20 remove_watcher() i=7 2024-03-04T18:06:00.901 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:00.901 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:00.901 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-commit-target-layout id: 797b67a4-b876-4d04-b122-10afd858d465.4453.26 num_objects: 19 size_kb: 1 num_shards: 12 2024-03-04T18:06:00.901 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:00.903 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:00.903 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:00.924 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:00.924 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:01.320 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:01.320 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:01.705 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with EIO injected at do_reshard 2024-03-04T18:06:01.705 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:01.710 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:01.710 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:01.857 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:01.858 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:01.943 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:01.943 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.026 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.026 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.112 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.112 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.197 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.198 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.284 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.284 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.374 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.374 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.461 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.461 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.548 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.548 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.633 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.633 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.717 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.717 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.807 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.807 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.896 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.896 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:02.993 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:02.993 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:03.074 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:03.074 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:03.159 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:03.159 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:03.246 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:03.247 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:03.333 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:03.333 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:03.420 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:03.420 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:03.507 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:03.507 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:03.589 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:03.590 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:03.677 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-03-04T18:06:03.710 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:03.710 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:03.755 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.758+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.757 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 20 realm 2024-03-04T18:06:03.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:03.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:03.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.762+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.766+0000 7eff71701980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:03.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.766+0000 7eff71701980 20 searching for the correct realm 2024-03-04T18:06:03.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.770+0000 7eff6fa1a640 0 -- 172.21.15.47:0/102153273 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7eff58008050 msgr2=0x7eff58083050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:03.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.778+0000 7eff6f219640 0 -- 172.21.15.47:0/102153273 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7eff680ef250 msgr2=0x7eff680edeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:03.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:03.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:03.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:03.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:03.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:03.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:03.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:03.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:03.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 zone default found 2024-03-04T18:06:03.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 4 Realm: () 2024-03-04T18:06:03.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:03.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:03.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:03.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 zonegroup default 2024-03-04T18:06:03.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.786+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.790+0000 7eff71701980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.787 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.790+0000 7eff71701980 20 started sync module instance, tier type = 2024-03-04T18:06:03.788 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.790+0000 7eff71701980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:03.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.806+0000 7eff71701980 20 add_watcher() i=0 2024-03-04T18:06:03.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.806+0000 7eff71701980 20 add_watcher() i=1 2024-03-04T18:06:03.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.806+0000 7eff71701980 20 add_watcher() i=2 2024-03-04T18:06:03.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.806+0000 7eff71701980 20 add_watcher() i=3 2024-03-04T18:06:03.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.806+0000 7eff71701980 20 add_watcher() i=4 2024-03-04T18:06:03.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.806+0000 7eff71701980 20 add_watcher() i=5 2024-03-04T18:06:03.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.810+0000 7eff71701980 20 add_watcher() i=6 2024-03-04T18:06:03.806 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.810+0000 7eff71701980 20 add_watcher() i=7 2024-03-04T18:06:03.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.810+0000 7eff71701980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:03.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.814+0000 7efe25fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:03.810 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.814+0000 7eff71701980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:03.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.814+0000 7eff71701980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:03.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.814+0000 7eff71701980 5 note: GC not initialized 2024-03-04T18:06:03.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.814+0000 7efe24fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:03.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.814+0000 7eff71701980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:03.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.814+0000 7eff71701980 20 Filter name: none 2024-03-04T18:06:03.812 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.814+0000 7efd9e7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:03.812 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.814+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.818+0000 7eff71701980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:03.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.818+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.818+0000 7eff71701980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:03.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.818+0000 7eff71701980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:03.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.818+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.818+0000 7eff71701980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:03.814 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.818+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.818+0000 7eff71701980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:03.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.818+0000 7eff71701980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:03.816 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.822+0000 7eff71701980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.822+0000 7eff71701980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.822+0000 7efe25fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:03.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.830+0000 7eff71701980 20 remove_watcher() i=0 2024-03-04T18:06:03.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.830+0000 7eff71701980 2 removed watcher, disabling cache 2024-03-04T18:06:03.826 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.830+0000 7eff71701980 20 remove_watcher() i=1 2024-03-04T18:06:03.828 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.834+0000 7eff71701980 20 remove_watcher() i=2 2024-03-04T18:06:03.831 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.834+0000 7eff71701980 20 remove_watcher() i=3 2024-03-04T18:06:03.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.838+0000 7eff71701980 20 remove_watcher() i=4 2024-03-04T18:06:03.835 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.838+0000 7eff71701980 20 remove_watcher() i=5 2024-03-04T18:06:03.838 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.842+0000 7eff71701980 20 remove_watcher() i=6 2024-03-04T18:06:03.840 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.846+0000 7eff71701980 20 remove_watcher() i=7 2024-03-04T18:06:03.855 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:03.855 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:03.855 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-do-reshard id: 797b67a4-b876-4d04-b122-10afd858d465.4453.27 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:06:03.855 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:03.855 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at do_reshard 2024-03-04T18:06:03.892 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:03.892 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:03.936 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.938+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.942+0000 7f8fb57b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.942+0000 7f8fb57b3980 20 realm 2024-03-04T18:06:03.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.942+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.942+0000 7f8fb57b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.942+0000 7f8fb57b3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:03.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.942+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.942+0000 7f8fb57b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:03.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.942+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.946+0000 7f8fb57b3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:03.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.946+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:03.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.946+0000 7f8fb57b3980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:03.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.946+0000 7f8fb57b3980 20 searching for the correct realm 2024-03-04T18:06:03.945 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.950+0000 7f8fb2aca640 0 -- 172.21.15.47:0/204716632 >> [v2:172.21.15.152:6808/929675582,v1:172.21.15.152:6809/929675582] conn(0x55cc0dc24e70 msgr2=0x55cc0dc27310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:03.952 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:03.958+0000 7f8fb32cb640 0 -- 172.21.15.47:0/204716632 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f8fac004040 msgr2=0x7f8fac112920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:04.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.174+0000 7f8fb57b3980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:04.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.174+0000 7f8fb57b3980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:04.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.174+0000 7f8fb57b3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:04.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.174+0000 7f8fb57b3980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:04.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.174+0000 7f8fb57b3980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:04.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.174+0000 7f8fb57b3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:04.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.174+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:04.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:04.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 zone default found 2024-03-04T18:06:04.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 4 Realm: () 2024-03-04T18:06:04.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:04.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:04.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:04.175 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 zonegroup default 2024-03-04T18:06:04.176 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.176 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.176 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.176 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.176 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.178+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.182+0000 7f8fb57b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.182+0000 7f8fb57b3980 20 started sync module instance, tier type = 2024-03-04T18:06:04.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.182+0000 7f8fb57b3980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:04.200 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.206+0000 7f8fb57b3980 20 add_watcher() i=0 2024-03-04T18:06:04.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.206+0000 7f8fb57b3980 20 add_watcher() i=1 2024-03-04T18:06:04.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.206+0000 7f8fb57b3980 20 add_watcher() i=2 2024-03-04T18:06:04.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.206+0000 7f8fb57b3980 20 add_watcher() i=3 2024-03-04T18:06:04.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.206+0000 7f8fb57b3980 20 add_watcher() i=4 2024-03-04T18:06:04.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.206+0000 7f8fb57b3980 20 add_watcher() i=5 2024-03-04T18:06:04.201 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.206+0000 7f8fb57b3980 20 add_watcher() i=6 2024-03-04T18:06:04.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.206+0000 7f8fb57b3980 20 add_watcher() i=7 2024-03-04T18:06:04.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.206+0000 7f8fb57b3980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:04.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.210+0000 7f8e69fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:04.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.210+0000 7f8fb57b3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:04.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.210+0000 7f8fb57b3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:04.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 5 note: GC not initialized 2024-03-04T18:06:04.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8e68fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:04.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:04.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 20 Filter name: none 2024-03-04T18:06:04.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8de27fc640 20 reqs_thread_entry: start 2024-03-04T18:06:04.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-03-04T18:06:04.211 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 : miss 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:04.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.214+0000 7f8fb57b3980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 2024-03-04T18:06:04.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.226+0000 7f8fb57b3980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.226+0000 7f8fb57b3980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.226+0000 7f8fb57b3980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.226+0000 7f8fb57b3980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27, ofs0, ns] 2024-03-04T18:06:04.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.226+0000 7f8e96ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595117 cookie 94334892550080 notifier 6452 bl.length()=873 2024-03-04T18:06:04.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.226+0000 7f8e96ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.226+0000 7f8e96ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.226+0000 7f8e96ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.230+0000 7f8fb57b3980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): orig_sources= new_sources= 2024-03-04T18:06:04.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.230+0000 7f8fb57b3980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): potential sources added=[] removed=[] 2024-03-04T18:06:04.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.230+0000 7f8fb57b3980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): orig_dests= new_dests= 2024-03-04T18:06:04.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.230+0000 7f8fb57b3980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): potential dests added=[] removed=[] 2024-03-04T18:06:04.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.238+0000 7f8fb57b3980 -1 Injecting error=-5 at location=do_reshard 2024-03-04T18:06:04.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.250+0000 7f8fb57b3980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:04.246 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.250+0000 7f8fb57b3980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 2024-03-04T18:06:04.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8fb57b3980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8fb57b3980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.248 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8fb57b3980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8fb57b3980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27, ofs0, ns] 2024-03-04T18:06:04.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8e967fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595118 cookie 94334892550080 notifier 6452 bl.length()=841 2024-03-04T18:06:04.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8e967fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8e967fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8e967fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8fb57b3980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): orig_sources= new_sources= 2024-03-04T18:06:04.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8fb57b3980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): potential sources added=[] removed=[] 2024-03-04T18:06:04.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8fb57b3980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): orig_dests= new_dests= 2024-03-04T18:06:04.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8fb57b3980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): potential dests added=[] removed=[] 2024-03-04T18:06:04.252 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.254+0000 7f8fb57b3980 1 execute INFO: reshard of bucket "error-at-do-reshard" canceled due to errors 2024-03-04T18:06:04.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.258+0000 7f8e69fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:04.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.266+0000 7f8fb57b3980 20 remove_watcher() i=0 2024-03-04T18:06:04.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.266+0000 7f8fb57b3980 2 removed watcher, disabling cache 2024-03-04T18:06:04.264 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.270+0000 7f8fb57b3980 20 remove_watcher() i=1 2024-03-04T18:06:04.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.270+0000 7f8fb57b3980 20 remove_watcher() i=2 2024-03-04T18:06:04.269 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.274+0000 7f8fb57b3980 20 remove_watcher() i=3 2024-03-04T18:06:04.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.278+0000 7f8fb57b3980 20 remove_watcher() i=4 2024-03-04T18:06:04.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.282+0000 7f8fb57b3980 20 remove_watcher() i=5 2024-03-04T18:06:04.279 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.282+0000 7f8fb57b3980 20 remove_watcher() i=6 2024-03-04T18:06:04.281 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.286+0000 7f8fb57b3980 20 remove_watcher() i=7 2024-03-04T18:06:04.298 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-03-04T18:06:04.330 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:04.330 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:04.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.378+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.378+0000 7f1a8e2a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.378+0000 7f1a8e2a1980 20 realm 2024-03-04T18:06:04.376 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.378+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.382+0000 7f1a8e2a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.382+0000 7f1a8e2a1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:04.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.382+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.382+0000 7f1a8e2a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.382+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.382+0000 7f1a8e2a1980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:04.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.382+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.382+0000 7f1a8e2a1980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:04.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.382+0000 7f1a8e2a1980 20 searching for the correct realm 2024-03-04T18:06:04.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.390+0000 7f1a8b5b8640 0 -- 172.21.15.47:0/3687268877 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f1a8006fe20 msgr2=0x7f1a80063660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:04.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.394+0000 7f1a8c5ba640 0 -- 172.21.15.47:0/3687268877 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55646a7e4220 msgr2=0x55646a7e6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:04.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.402+0000 7f1a8e2a1980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:04.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.402+0000 7f1a8e2a1980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:04.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.402+0000 7f1a8e2a1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:04.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.402+0000 7f1a8e2a1980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:04.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.402+0000 7f1a8e2a1980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:04.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.402+0000 7f1a8e2a1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:04.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.402+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.404 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 zone default found 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 4 Realm: () 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 zonegroup default 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 started sync module instance, tier type = 2024-03-04T18:06:04.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.406+0000 7f1a8e2a1980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:04.422 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.426+0000 7f1a8e2a1980 20 add_watcher() i=0 2024-03-04T18:06:04.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.426+0000 7f1a8e2a1980 20 add_watcher() i=1 2024-03-04T18:06:04.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.426+0000 7f1a8e2a1980 20 add_watcher() i=2 2024-03-04T18:06:04.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.426+0000 7f1a8e2a1980 20 add_watcher() i=3 2024-03-04T18:06:04.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.426+0000 7f1a8e2a1980 20 add_watcher() i=4 2024-03-04T18:06:04.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.426+0000 7f1a8e2a1980 20 add_watcher() i=5 2024-03-04T18:06:04.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.426+0000 7f1a8e2a1980 20 add_watcher() i=6 2024-03-04T18:06:04.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.430+0000 7f1a8e2a1980 20 add_watcher() i=7 2024-03-04T18:06:04.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.430+0000 7f1a8e2a1980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:04.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1941fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:04.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:04.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:04.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 5 note: GC not initialized 2024-03-04T18:06:04.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1940fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:04.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:04.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 Filter name: none 2024-03-04T18:06:04.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f18ba7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:04.430 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:04.431 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.432 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:04.432 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:04.432 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.434+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.438+0000 7f1a8e2a1980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:04.433 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.438+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.438+0000 7f1a8e2a1980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:04.434 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.438+0000 7f1a8e2a1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:04.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.438+0000 7f1a8e2a1980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.436 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.438+0000 7f1a8e2a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.437 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.442+0000 7f1941fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:04.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.446+0000 7f1a8e2a1980 20 remove_watcher() i=0 2024-03-04T18:06:04.441 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.446+0000 7f1a8e2a1980 2 removed watcher, disabling cache 2024-03-04T18:06:04.443 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.446+0000 7f1a8e2a1980 20 remove_watcher() i=1 2024-03-04T18:06:04.448 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.450+0000 7f1a8e2a1980 20 remove_watcher() i=2 2024-03-04T18:06:04.450 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.454+0000 7f1a8e2a1980 20 remove_watcher() i=3 2024-03-04T18:06:04.452 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.454+0000 7f1a8e2a1980 20 remove_watcher() i=4 2024-03-04T18:06:04.456 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.458+0000 7f1a8e2a1980 20 remove_watcher() i=5 2024-03-04T18:06:04.462 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.466+0000 7f1a8e2a1980 20 remove_watcher() i=6 2024-03-04T18:06:04.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.470+0000 7f1a8e2a1980 20 remove_watcher() i=7 2024-03-04T18:06:04.482 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:04.482 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:04.482 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-do-reshard id: 797b67a4-b876-4d04-b122-10afd858d465.4453.27 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:06:04.482 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:04.483 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:04.483 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:04.519 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:04.519 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:04.578 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:06:04.614 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:04.615 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:04.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.662+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 20 realm 2024-03-04T18:06:04.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:04.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:04.664 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.666+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.670+0000 7f949b439980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:04.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.670+0000 7f949b439980 20 searching for the correct realm 2024-03-04T18:06:04.673 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.678+0000 7f9499752640 0 -- 172.21.15.47:0/3593017445 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x560adefdb200 msgr2=0x560adefdd660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:04.885 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.890+0000 7f9499752640 0 -- 172.21.15.47:0/3593017445 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x7f9364075c60 msgr2=0x7f9364078090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:04.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:04.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:04.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:04.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:04.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:04.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:04.897 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:04.898 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:04.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 zone default found 2024-03-04T18:06:04.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 4 Realm: () 2024-03-04T18:06:04.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:04.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:04.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:04.899 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 zonegroup default 2024-03-04T18:06:04.900 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.900 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.900 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.902+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.906+0000 7f949b439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.906+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.901 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.906+0000 7f949b439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.906+0000 7f949b439980 20 started sync module instance, tier type = 2024-03-04T18:06:04.902 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.906+0000 7f949b439980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:04.923 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.926+0000 7f949b439980 20 add_watcher() i=0 2024-03-04T18:06:04.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.926+0000 7f949b439980 20 add_watcher() i=1 2024-03-04T18:06:04.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.926+0000 7f949b439980 20 add_watcher() i=2 2024-03-04T18:06:04.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.926+0000 7f949b439980 20 add_watcher() i=3 2024-03-04T18:06:04.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.926+0000 7f949b439980 20 add_watcher() i=4 2024-03-04T18:06:04.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.926+0000 7f949b439980 20 add_watcher() i=5 2024-03-04T18:06:04.924 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.926+0000 7f949b439980 20 add_watcher() i=6 2024-03-04T18:06:04.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.930+0000 7f949b439980 20 add_watcher() i=7 2024-03-04T18:06:04.926 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.930+0000 7f949b439980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:04.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.934+0000 7f934efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:04.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.934+0000 7f949b439980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:04.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.934+0000 7f949b439980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:04.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.934+0000 7f949b439980 5 note: GC not initialized 2024-03-04T18:06:04.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.934+0000 7f934dfe3640 20 reqs_thread_entry: start 2024-03-04T18:06:04.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.934+0000 7f949b439980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:04.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.934+0000 7f949b439980 20 Filter name: none 2024-03-04T18:06:04.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f92bf7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:04.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-03-04T18:06:04.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:04.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-03-04T18:06:04.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-03-04T18:06:04.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 : miss 2024-03-04T18:06:04.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:04.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.934 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:04.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.935 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.938+0000 7f949b439980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 2024-03-04T18:06:04.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.950+0000 7f949b439980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.950+0000 7f949b439980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.950+0000 7f949b439980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.946 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.950+0000 7f949b439980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27, ofs0, ns] 2024-03-04T18:06:04.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.950+0000 7f936f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595119 cookie 94604690897152 notifier 6900 bl.length()=873 2024-03-04T18:06:04.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.950+0000 7f936f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.950+0000 7f936f7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.947 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.950+0000 7f936f7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.950 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.954+0000 7f949b439980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): orig_sources= new_sources= 2024-03-04T18:06:04.950 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.954+0000 7f949b439980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): potential sources added=[] removed=[] 2024-03-04T18:06:04.950 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.954+0000 7f949b439980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): orig_dests= new_dests= 2024-03-04T18:06:04.950 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.954+0000 7f949b439980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): potential dests added=[] removed=[] 2024-03-04T18:06:04.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.0 2024-03-04T18:06:04.956 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.0 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.1 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.1 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.2 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.2 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.3 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.3 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.4 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.4 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.5 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.5 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.6 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.6 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.7 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.7 2024-03-04T18:06:04.957 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.8 2024-03-04T18:06:04.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.8 2024-03-04T18:06:04.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.9 2024-03-04T18:06:04.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.9 2024-03-04T18:06:04.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.10 2024-03-04T18:06:04.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.10 2024-03-04T18:06:04.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.11 2024-03-04T18:06:04.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1.11 2024-03-04T18:06:04.958 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.0 2024-03-04T18:06:04.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.1 2024-03-04T18:06:04.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.2 2024-03-04T18:06:04.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.3 2024-03-04T18:06:04.959 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.962+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.4 2024-03-04T18:06:04.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.966+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.5 2024-03-04T18:06:04.961 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.966+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.6 2024-03-04T18:06:04.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.966+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.7 2024-03-04T18:06:04.963 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.966+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.8 2024-03-04T18:06:04.965 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.970+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.9 2024-03-04T18:06:04.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.970+0000 7f949b439980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.27.10 2024-03-04T18:06:04.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.978+0000 7f949b439980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:04.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.978+0000 7f949b439980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 2024-03-04T18:06:04.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.982+0000 7f949b439980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.977 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.982+0000 7f949b439980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.982+0000 7f949b439980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.982+0000 7f949b439980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27, ofs0, ns] 2024-03-04T18:06:04.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.982+0000 7f936ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595120 cookie 94604690897152 notifier 6900 bl.length()=841 2024-03-04T18:06:04.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.982+0000 7f936ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 info.flags=0x17 2024-03-04T18:06:04.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.982+0000 7f936ffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.27 to cache LRU end 2024-03-04T18:06:04.978 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.982+0000 7f936ffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:04.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.986+0000 7f949b439980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): orig_sources= new_sources= 2024-03-04T18:06:04.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.986+0000 7f949b439980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): potential sources added=[] removed=[] 2024-03-04T18:06:04.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.986+0000 7f949b439980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): orig_dests= new_dests= 2024-03-04T18:06:04.980 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.986+0000 7f949b439980 20 handle_bi_update(): bucket=:error-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.27]): potential dests added=[] removed=[] 2024-03-04T18:06:04.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.990+0000 7f949b439980 1 execute INFO: reshard of bucket "error-at-do-reshard" completed successfully 2024-03-04T18:06:04.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.994+0000 7f934efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:04.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.998+0000 7f949b439980 20 remove_watcher() i=0 2024-03-04T18:06:04.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:04.998+0000 7f949b439980 2 removed watcher, disabling cache 2024-03-04T18:06:04.998 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.002+0000 7f949b439980 20 remove_watcher() i=1 2024-03-04T18:06:05.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.002+0000 7f949b439980 20 remove_watcher() i=2 2024-03-04T18:06:05.002 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.006+0000 7f949b439980 20 remove_watcher() i=3 2024-03-04T18:06:05.004 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.010+0000 7f949b439980 20 remove_watcher() i=4 2024-03-04T18:06:05.006 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.010+0000 7f949b439980 20 remove_watcher() i=5 2024-03-04T18:06:05.009 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.014+0000 7f949b439980 20 remove_watcher() i=6 2024-03-04T18:06:05.012 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.014+0000 7f949b439980 20 remove_watcher() i=7 2024-03-04T18:06:05.033 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-03-04T18:06:05.067 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:05.067 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:05.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.114+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.118+0000 7f352db7b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.118+0000 7f352db7b980 20 realm 2024-03-04T18:06:05.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.118+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.118+0000 7f352db7b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.118+0000 7f352db7b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:05.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.118+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.118+0000 7f352db7b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.118+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.122+0000 7f352db7b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:05.116 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.122+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.122+0000 7f352db7b980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:05.117 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.122+0000 7f352db7b980 20 searching for the correct realm 2024-03-04T18:06:05.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.126+0000 7f3523fff640 0 -- 172.21.15.47:0/1550948156 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f3514015940 msgr2=0x7f351401c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:05.337 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.342+0000 7f3523fff640 0 -- 172.21.15.47:0/1550948156 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x56375a42a9b0 msgr2=0x56375a42ce50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:05.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:05.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:05.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:05.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:05.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:05.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:05.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.350 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:05.351 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:05.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 zone default found 2024-03-04T18:06:05.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 4 Realm: () 2024-03-04T18:06:05.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:05.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:05.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:05.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 zonegroup default 2024-03-04T18:06:05.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.354+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.358+0000 7f352db7b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.358+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.358+0000 7f352db7b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.358+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.358+0000 7f352db7b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.358+0000 7f352db7b980 20 started sync module instance, tier type = 2024-03-04T18:06:05.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.358+0000 7f352db7b980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:05.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.382+0000 7f352db7b980 20 add_watcher() i=0 2024-03-04T18:06:05.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.382+0000 7f352db7b980 20 add_watcher() i=1 2024-03-04T18:06:05.378 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.382+0000 7f352db7b980 20 add_watcher() i=2 2024-03-04T18:06:05.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.382+0000 7f352db7b980 20 add_watcher() i=3 2024-03-04T18:06:05.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.382+0000 7f352db7b980 20 add_watcher() i=4 2024-03-04T18:06:05.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.382+0000 7f352db7b980 20 add_watcher() i=5 2024-03-04T18:06:05.379 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.382+0000 7f352db7b980 20 add_watcher() i=6 2024-03-04T18:06:05.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.386+0000 7f352db7b980 20 add_watcher() i=7 2024-03-04T18:06:05.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.386+0000 7f352db7b980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:05.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.390+0000 7f33e27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:05.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.390+0000 7f352db7b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:05.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.390+0000 7f352db7b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:05.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.390+0000 7f352db7b980 5 note: GC not initialized 2024-03-04T18:06:05.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.390+0000 7f33e17e2640 20 reqs_thread_entry: start 2024-03-04T18:06:05.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.390+0000 7f352db7b980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:05.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.390+0000 7f352db7b980 20 Filter name: none 2024-03-04T18:06:05.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.390+0000 7f33da7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:05.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.390+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.394+0000 7f352db7b980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:05.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.394+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.394+0000 7f352db7b980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:05.389 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.394+0000 7f352db7b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:05.390 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.394+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.394+0000 7f352db7b980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:05.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.394+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.394+0000 7f352db7b980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:05.391 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.394+0000 7f352db7b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:05.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.398+0000 7f352db7b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.398+0000 7f352db7b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.398+0000 7f33e27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:05.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.406+0000 7f352db7b980 20 remove_watcher() i=0 2024-03-04T18:06:05.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.406+0000 7f352db7b980 2 removed watcher, disabling cache 2024-03-04T18:06:05.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.406+0000 7f352db7b980 20 remove_watcher() i=1 2024-03-04T18:06:05.404 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.410+0000 7f352db7b980 20 remove_watcher() i=2 2024-03-04T18:06:05.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.410+0000 7f352db7b980 20 remove_watcher() i=3 2024-03-04T18:06:05.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.414+0000 7f352db7b980 20 remove_watcher() i=4 2024-03-04T18:06:05.412 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.414+0000 7f352db7b980 20 remove_watcher() i=5 2024-03-04T18:06:05.414 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.418+0000 7f352db7b980 20 remove_watcher() i=6 2024-03-04T18:06:05.417 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:05.422+0000 7f352db7b980 20 remove_watcher() i=7 2024-03-04T18:06:05.439 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:05.439 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:05.440 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: error-at-do-reshard id: 797b67a4-b876-4d04-b122-10afd858d465.4453.27 num_objects: 19 size_kb: 1 num_shards: 12 2024-03-04T18:06:05.440 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:05.441 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:05.441 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:05.462 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:05.462 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:05.926 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:05.926 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:06.298 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: reshard bucket with abort at do_reshard 2024-03-04T18:06:06.298 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:06.301 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.301 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:06.431 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.431 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:06.517 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.517 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:06.603 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.603 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:06.688 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.688 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:06.774 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.774 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:06.861 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.861 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:06.945 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.945 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.030 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.030 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.118 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.118 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.205 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.205 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.294 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.294 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.380 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.380 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.466 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.467 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.555 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.555 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.643 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.643 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.728 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.728 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.813 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.813 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.900 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.900 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:07.986 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.986 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:08.071 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:08.071 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:08.159 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:08.159 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:08.244 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-03-04T18:06:08.280 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:08.280 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:08.325 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.330+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.330+0000 7fea36a93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.330+0000 7fea36a93980 20 realm 2024-03-04T18:06:08.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.330+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.334+0000 7fea36a93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.334+0000 7fea36a93980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.334+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.334+0000 7fea36a93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.334+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.334+0000 7fea36a93980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.334+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.334+0000 7fea36a93980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:08.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.334+0000 7fea36a93980 20 searching for the correct realm 2024-03-04T18:06:08.338 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.342+0000 7fea34dac640 0 -- 172.21.15.47:0/3466455719 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7fea280c0400 msgr2=0x7fea28009590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.344 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.346+0000 7fea345ab640 0 -- 172.21.15.47:0/3466455719 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fea2c028040 msgr2=0x7fea2c03a960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:08.353 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:08.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:08.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 zone default found 2024-03-04T18:06:08.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 4 Realm: () 2024-03-04T18:06:08.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:08.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:08.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:08.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 zonegroup default 2024-03-04T18:06:08.355 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 started sync module instance, tier type = 2024-03-04T18:06:08.356 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.358+0000 7fea36a93980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.378+0000 7fea36a93980 20 add_watcher() i=0 2024-03-04T18:06:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.378+0000 7fea36a93980 20 add_watcher() i=1 2024-03-04T18:06:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.378+0000 7fea36a93980 20 add_watcher() i=2 2024-03-04T18:06:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.378+0000 7fea36a93980 20 add_watcher() i=3 2024-03-04T18:06:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.378+0000 7fea36a93980 20 add_watcher() i=4 2024-03-04T18:06:08.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.378+0000 7fea36a93980 20 add_watcher() i=5 2024-03-04T18:06:08.375 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.378+0000 7fea36a93980 20 add_watcher() i=6 2024-03-04T18:06:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.382+0000 7fea36a93980 20 add_watcher() i=7 2024-03-04T18:06:08.377 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.382+0000 7fea36a93980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:08.380 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fe8eafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:08.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:08.381 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 5 note: GC not initialized 2024-03-04T18:06:08.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fe8e9fdb640 20 reqs_thread_entry: start 2024-03-04T18:06:08.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:08.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 Filter name: none 2024-03-04T18:06:08.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fe85b7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:08.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:08.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:08.383 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:08.384 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.386+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.390+0000 7fea36a93980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:08.385 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.390+0000 7fea36a93980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:08.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.390+0000 7fea36a93980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.390+0000 7fea36a93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.387 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.390+0000 7fe8eafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.394+0000 7fea36a93980 20 remove_watcher() i=0 2024-03-04T18:06:08.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.394+0000 7fea36a93980 2 removed watcher, disabling cache 2024-03-04T18:06:08.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.398+0000 7fea36a93980 20 remove_watcher() i=1 2024-03-04T18:06:08.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.398+0000 7fea36a93980 20 remove_watcher() i=2 2024-03-04T18:06:08.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.402+0000 7fea36a93980 20 remove_watcher() i=3 2024-03-04T18:06:08.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.406+0000 7fea36a93980 20 remove_watcher() i=4 2024-03-04T18:06:08.402 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.406+0000 7fea36a93980 20 remove_watcher() i=5 2024-03-04T18:06:08.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.410+0000 7fea36a93980 20 remove_watcher() i=6 2024-03-04T18:06:08.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.410+0000 7fea36a93980 20 remove_watcher() i=7 2024-03-04T18:06:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.423 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:08.423 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-do-reshard id: 797b67a4-b876-4d04-b122-10afd858d465.4453.28 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:06:08.423 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.423 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2024-03-04T18:06:08.455 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:08.455 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:08.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.502+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 realm 2024-03-04T18:06:08.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:08.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:08.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 searching for the correct realm 2024-03-04T18:06:08.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.514+0000 7fdfbcb41640 0 -- 172.21.15.47:0/3463461626 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fdfb40659d0 msgr2=0x7fdfb408b820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.518+0000 7fdfbd342640 0 -- 172.21.15.47:0/3463461626 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fdfb8124c20 msgr2=0x7fdfb80f7b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:08.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:08.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 zone default found 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 4 Realm: () 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 zonegroup default 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 started sync module instance, tier type = 2024-03-04T18:06:08.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:08.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=0 2024-03-04T18:06:08.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=1 2024-03-04T18:06:08.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=2 2024-03-04T18:06:08.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=3 2024-03-04T18:06:08.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=4 2024-03-04T18:06:08.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=5 2024-03-04T18:06:08.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=6 2024-03-04T18:06:08.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.554+0000 7fdfbf82a980 20 add_watcher() i=7 2024-03-04T18:06:08.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.554+0000 7fdfbf82a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:08.552 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.554+0000 7fde73fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:08.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:08.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 note: GC not initialized 2024-03-04T18:06:08.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fde72fe5640 20 reqs_thread_entry: start 2024-03-04T18:06:08.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:08.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 Filter name: none 2024-03-04T18:06:08.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fde70fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:08.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-03-04T18:06:08.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:08.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-03-04T18:06:08.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-03-04T18:06:08.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 : miss 2024-03-04T18:06:08.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:08.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:08.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:08.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:08.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:08.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 2024-03-04T18:06:08.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:08.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:08.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:08.567 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28, ofs0, ns] 2024-03-04T18:06:08.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595124 cookie 93855323373376 notifier 6539 bl.length()=873 2024-03-04T18:06:08.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:08.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:08.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:08.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_sources= new_sources= 2024-03-04T18:06:08.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential sources added=[] removed=[] 2024-03-04T18:06:08.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_dests= new_dests= 2024-03-04T18:06:08.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential dests added=[] removed=[] 2024-03-04T18:06:08.576 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fdfbf82a980 time 2024-03-04T18:06:08.583168+0000 2024-03-04T18:06:08.576 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fdfc30ffd7b] 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fdfc30fff2f] 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x555c626ed281] 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x555c626f9d5d] 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fdfc1f6ad90] 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr:*** Caught signal (Aborted) ** 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7fdfbf82a980 thread_name:radosgw-admin 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.582+0000 7fdfbf82a980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fdfbf82a980 time 2024-03-04T18:06:08.583168+0000 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fdfc30ffd7b] 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fdfc30fff2f] 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x555c626ed281] 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x555c626f9d5d] 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fdfc1f6ad90] 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fdfc1f83520] 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fdfc30ffdd5] 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fdfc30fff2f] 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x555c626ed281] 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x555c626f9d5d] 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fdfc1f6ad90] 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.582+0000 7fdfbf82a980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7fdfbf82a980 thread_name:radosgw-admin 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fdfc1f83520] 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fdfc30ffdd5] 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fdfc30fff2f] 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x555c626ed281] 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x555c626f9d5d] 2024-03-04T18:06:08.596 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fdfc1f6ad90] 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr:--- begin dump of recent events --- 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr: -499> 2024-03-04T18:06:08.482+0000 7fdfbcb41640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr: -498> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr: -497> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr: -496> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: dump: 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.597 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: -495> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: -494> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: -493> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6527 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: -492> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: -491> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: get_monmap_and_config success 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: -490> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 4 set_mon_vals no callback set 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: -489> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: -488> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr: -487> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: dump: 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.598 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr: -486> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: shutdown 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr: -485> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 asok(0x555c6507a130) unregister_commands rotate-key 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr: -484> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: build_initial_monmap 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr: -483> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 build_initial for_mkfs: 0 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr: -482> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: monmap: 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr: -481> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding auth protocol: cephx 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr: -480> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding auth protocol: cephx 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr: -479> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding auth protocol: cephx 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr: -478> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr: -477> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr: -476> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi047.stderr: -475> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -474> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -473> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -472> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -471> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -470> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -469> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -468> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -467> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -466> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -465> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 librados: starting msgr at 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -464> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 librados: starting objecter 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -463> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command objecter_requests hook 0x555c6508f130 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -462> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 librados: setting wanted keys 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -461> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 librados: calling monclient init 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -460> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: init 2024-03-04T18:06:08.601 INFO:tasks.workunit.client.0.smithi047.stderr: -459> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding auth protocol: cephx 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -458> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding auth protocol: cephx 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -457> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding auth protocol: cephx 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -456> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -455> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -454> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -453> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -452> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -451> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -450> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -449> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -448> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi047.stderr: -447> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -446> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -445> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -444> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -443> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -442> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command rotate-key hook 0x555c65261e48 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -441> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _reopen_session rank -1 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -440> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -439> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-b con 0x555c651af120 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -438> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-a con 0x555c651afdc0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -437> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-c con 0x555c65290bf0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -436> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -435> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -434> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -433> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): _renew_subs 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -432> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): authenticate will time out at 2539.660087s 2024-03-04T18:06:08.603 INFO:tasks.workunit.client.0.smithi047.stderr: -431> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request con 0x555c651afdc0 auth_method 0 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -430> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -429> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -428> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -427> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request con 0x555c65290bf0 auth_method 0 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -426> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -425> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -424> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -423> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -422> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -421> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -420> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -419> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -418> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi047.stderr: -417> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient(hunting): get_auth_request con 0x555c651af120 auth_method 0 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -416> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -415> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -414> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -413> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_done global_id 6530 payload 824 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -412> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient: _finish_hunting 0 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -411> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 1 monclient: found mon.noname-c 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -410> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -409> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -408> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr: -407> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: dump: 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.605 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr: -406> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr: -405> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr: -404> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr: -403> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr: -402> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr: -401> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: dump: 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -400> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6530 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -399> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -398> 2024-03-04T18:06:08.486+0000 7fdeab606640 4 set_mon_vals no callback set 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -397> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -396> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -395> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -394> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 1 librados: init done 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -393> 2024-03-04T18:06:08.486+0000 7fdea9e03640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -392> 2024-03-04T18:06:08.486+0000 7fdea9e03640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -391> 2024-03-04T18:06:08.486+0000 7fdea9e03640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -390> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fdea4065940 auth_method 0 2024-03-04T18:06:08.607 INFO:tasks.workunit.client.0.smithi047.stderr: -389> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x555c65265ce0 auth_method 0 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr: -388> 2024-03-04T18:06:08.490+0000 7fdfbd342640 10 monclient: get_auth_request con 0x555c652ac910 auth_method 0 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr: -387> 2024-03-04T18:06:08.490+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x555c652b19a0 auth_method 0 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr: -386> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 10 monclient: build_initial_monmap 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr: -385> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 1 build_initial for_mkfs: 0 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr: -384> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 10 monclient: monmap: 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -383> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding auth protocol: cephx 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -382> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding auth protocol: cephx 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -381> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding auth protocol: cephx 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -380> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -379> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -378> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -377> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -376> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -375> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -374> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -373> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -372> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.609 INFO:tasks.workunit.client.0.smithi047.stderr: -371> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -370> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -369> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -368> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -367> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command objecter_requests cmddesc objecter_requests hook 0x555c652b9560 EEXIST 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -366> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: init 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -365> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding auth protocol: cephx 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -364> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding auth protocol: cephx 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -363> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding auth protocol: cephx 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -362> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -361> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -360> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -359> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -358> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -357> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -356> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi047.stderr: -355> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -354> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -353> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -352> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -351> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -350> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -349> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -348> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command rotate-key cmddesc rotate-key hook 0x555c652b4a90 EEXIST 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -347> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _reopen_session rank -1 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -346> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -345> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-b con 0x555c652dfcd0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -344> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-c con 0x555c652e00b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -343> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-a con 0x555c652db270 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -342> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -341> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -340> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.611 INFO:tasks.workunit.client.0.smithi047.stderr: -339> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): _renew_subs 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -338> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): authenticate will time out at 2539.669000s 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -337> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request con 0x555c652db270 auth_method 0 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -336> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -335> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -334> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -333> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request con 0x555c652e00b0 auth_method 0 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -332> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -331> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -330> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -329> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -328> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -327> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -326> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.612 INFO:tasks.workunit.client.0.smithi047.stderr: -325> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -324> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -323> 2024-03-04T18:06:08.494+0000 7fdfbd342640 10 monclient(hunting): get_auth_request con 0x555c652dfcd0 auth_method 0 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -322> 2024-03-04T18:06:08.494+0000 7fdfbd342640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -321> 2024-03-04T18:06:08.494+0000 7fdfbd342640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -320> 2024-03-04T18:06:08.494+0000 7fdfbd342640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -319> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_done global_id 6954 payload 824 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -318> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient: _finish_hunting 0 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -317> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 1 monclient: found mon.noname-a 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -316> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -315> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -314> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr: -313> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: dump: 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr: -312> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr: -311> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr: -310> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr: -309> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr: -308> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:06:08.614 INFO:tasks.workunit.client.0.smithi047.stderr: -307> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: dump: 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr: -306> 2024-03-04T18:06:08.494+0000 7fdfbe344640 4 set_mon_vals no callback set 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr: -305> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6954 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr: -304> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr: -303> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr: -302> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.615 INFO:tasks.workunit.client.0.smithi047.stderr: -301> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr: -300> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr: -299> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr: -298> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr: -297> 2024-03-04T18:06:08.498+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fde98065880 auth_method 0 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr: -296> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: get_monmap_and_config 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr: -295> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: build_initial_monmap 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr: -294> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 build_initial for_mkfs: 0 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr: -293> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: monmap: 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -292> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -291> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -290> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -289> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -288> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -287> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -286> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -285> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -284> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -283> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -282> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.617 INFO:tasks.workunit.client.0.smithi047.stderr: -281> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -280> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -279> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -278> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -277> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -276> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: init 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -275> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding auth protocol: cephx 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -274> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding auth protocol: cephx 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -273> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding auth protocol: cephx 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -272> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -271> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -270> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -269> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -268> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -267> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -266> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.618 INFO:tasks.workunit.client.0.smithi047.stderr: -265> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -264> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -263> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -262> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -261> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -260> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -259> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -258> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command rotate-key cmddesc rotate-key hook 0x7ffea8ca26b8 EEXIST 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -257> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: _reopen_session rank -1 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -256> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -255> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-b con 0x555c65331df0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -254> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-a con 0x555c652ffc10 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -253> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-c con 0x555c6532d7b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -252> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -251> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -250> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -249> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): _renew_subs 2024-03-04T18:06:08.619 INFO:tasks.workunit.client.0.smithi047.stderr: -248> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): authenticate will time out at 2539.673413s 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -247> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request con 0x555c652ffc10 auth_method 0 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -246> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -245> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -244> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -243> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request con 0x555c6532d7b0 auth_method 0 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -242> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -241> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -240> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -239> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -238> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -237> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -236> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -235> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.620 INFO:tasks.workunit.client.0.smithi047.stderr: -234> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr: -233> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_done global_id 6536 payload 274 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr: -232> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient: _finish_hunting 0 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr: -231> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 1 monclient: found mon.noname-c 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr: -230> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr: -229> 2024-03-04T18:06:08.498+0000 7fdfbd342640 10 monclient: get_auth_request con 0x555c65331df0 auth_method 0 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr: -228> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr: -227> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr: -226> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: dump: 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.621 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: -225> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: -224> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: -223> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6536 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: -222> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: -221> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: get_monmap_and_config success 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: -220> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 4 set_mon_vals no callback set 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: -219> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: -218> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr: -217> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: dump: 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.622 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr: -216> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: shutdown 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr: -215> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: build_initial_monmap 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr: -214> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 build_initial for_mkfs: 0 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr: -213> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: monmap: 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.623 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -212> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -211> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -210> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -209> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -208> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -207> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -206> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -205> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -204> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -203> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.624 INFO:tasks.workunit.client.0.smithi047.stderr: -202> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -201> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -200> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -199> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -198> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -197> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -196> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 librados: starting msgr at 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -195> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 librados: starting objecter 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -194> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command objecter_requests cmddesc objecter_requests hook 0x555c65328490 EEXIST 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -193> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 librados: setting wanted keys 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -192> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 librados: calling monclient init 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -191> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: init 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -190> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding auth protocol: cephx 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -189> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding auth protocol: cephx 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -188> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding auth protocol: cephx 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -187> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -186> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.625 INFO:tasks.workunit.client.0.smithi047.stderr: -185> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -184> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -183> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -182> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -181> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -180> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -179> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -178> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -177> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -176> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -175> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -174> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -173> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command rotate-key cmddesc rotate-key hook 0x555c652fcb38 EEXIST 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -172> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: _reopen_session rank -1 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -171> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -170> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-a con 0x555c652ffc10 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:06:08.626 INFO:tasks.workunit.client.0.smithi047.stderr: -169> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-c con 0x555c6532d7b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -168> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-b con 0x555c65331df0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -167> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -166> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -165> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -164> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): _renew_subs 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -163> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): authenticate will time out at 2539.675999s 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -162> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request con 0x555c6532d7b0 auth_method 0 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -161> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -160> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -159> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -158> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): get_auth_request con 0x555c652ffc10 auth_method 0 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -157> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -156> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -155> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.627 INFO:tasks.workunit.client.0.smithi047.stderr: -154> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -153> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -152> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -151> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -150> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -149> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -148> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_done global_id 6539 payload 824 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -147> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient: _finish_hunting 0 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -146> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 1 monclient: found mon.noname-c 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -145> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -144> 2024-03-04T18:06:08.502+0000 7fdfbdb43640 10 monclient: get_auth_request con 0x555c65331df0 auth_method 0 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -143> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -142> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr: -141> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: dump: 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.628 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: -140> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: -139> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: -138> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6539 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: -137> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: -136> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: -135> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: -134> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: -133> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:06:08.629 INFO:tasks.workunit.client.0.smithi047.stderr: -132> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: dump: 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr: -131> 2024-03-04T18:06:08.502+0000 7fdea0ff9640 4 set_mon_vals no callback set 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr: -130> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr: -129> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr: -128> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 1 librados: init done 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr: -127> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cr dump hook 0x555c65322af8 2024-03-04T18:06:08.630 INFO:tasks.workunit.client.0.smithi047.stderr: -126> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -125> 2024-03-04T18:06:08.502+0000 7fde8effd640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -124> 2024-03-04T18:06:08.502+0000 7fde8effd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -123> 2024-03-04T18:06:08.502+0000 7fde8effd640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -122> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fde88065940 auth_method 0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -121> 2024-03-04T18:06:08.502+0000 7fdfbdb43640 10 monclient: get_auth_request con 0x555c6534b3b0 auth_method 0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -120> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -119> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 realm 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -118> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -117> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -116> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -115> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -114> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -113> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -112> 2024-03-04T18:06:08.506+0000 7fdfbd342640 10 monclient: get_auth_request con 0x555c6534f620 auth_method 0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -111> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -110> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.631 INFO:tasks.workunit.client.0.smithi047.stderr: -109> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -108> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 searching for the correct realm 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -107> 2024-03-04T18:06:08.506+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x555c65354730 auth_method 0 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -106> 2024-03-04T18:06:08.510+0000 7fdfbdb43640 10 monclient: get_auth_request con 0x7fdfb4008070 auth_method 0 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -105> 2024-03-04T18:06:08.510+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fdfb0008040 auth_method 0 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -104> 2024-03-04T18:06:08.514+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x7fdfb40659d0 auth_method 0 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -103> 2024-03-04T18:06:08.514+0000 7fdfbcb41640 0 -- 172.21.15.47:0/3463461626 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fdfb40659d0 msgr2=0x7fdfb408b820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -102> 2024-03-04T18:06:08.514+0000 7fde8effd640 1 client.6539.objecter ms_handle_reset 0x7fdfb40659d0 session 0x7fdfb4003b20 osd.5 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -101> 2024-03-04T18:06:08.514+0000 7fde8effd640 10 monclient: _renew_subs 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -100> 2024-03-04T18:06:08.514+0000 7fde8effd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -99> 2024-03-04T18:06:08.514+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x7fde88075be0 auth_method 0 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -98> 2024-03-04T18:06:08.518+0000 7fdfbdb43640 10 monclient: get_auth_request con 0x7fdfb0051040 auth_method 0 2024-03-04T18:06:08.632 INFO:tasks.workunit.client.0.smithi047.stderr: -97> 2024-03-04T18:06:08.518+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fdfb8124c20 auth_method 0 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -96> 2024-03-04T18:06:08.518+0000 7fdfbd342640 0 -- 172.21.15.47:0/3463461626 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fdfb8124c20 msgr2=0x7fdfb80f7b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -95> 2024-03-04T18:06:08.518+0000 7fde8effd640 1 client.6539.objecter ms_handle_reset 0x7fdfb8124c20 session 0x7fdfb8107700 osd.3 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -94> 2024-03-04T18:06:08.522+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fdfb40659d0 auth_method 0 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -93> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -92> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -91> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -90> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -89> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -88> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -87> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -86> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -85> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -84> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -83> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi047.stderr: -82> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -81> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 zone default found 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -80> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 4 Realm: () 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -79> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -78> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -77> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -76> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 zonegroup default 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -75> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -74> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -73> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -72> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -71> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -70> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -69> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 started sync module instance, tier type = 2024-03-04T18:06:08.634 INFO:tasks.workunit.client.0.smithi047.stderr: -68> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -67> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=0 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -66> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=1 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -65> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=2 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -64> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=3 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -63> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=4 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -62> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=5 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -61> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=6 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -60> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 20 add_watcher() i=7 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -59> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -58> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding auth protocol: cephx 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -57> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding auth protocol: cephx 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -56> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding auth protocol: cephx 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -55> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -54> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -53> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.635 INFO:tasks.workunit.client.0.smithi047.stderr: -52> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -51> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -50> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -49> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -48> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -47> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -46> 2024-03-04T18:06:08.554+0000 7fde73fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -45> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -44> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -43> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -42> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -41> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -40> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -39> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cache list hook 0x555c65329c20 2024-03-04T18:06:08.636 INFO:tasks.workunit.client.0.smithi047.stderr: -38> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cache inspect hook 0x555c65329c20 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -37> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cache erase hook 0x555c65329c20 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -36> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cache zap hook 0x555c65329c20 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -35> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 note: GC not initialized 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -34> 2024-03-04T18:06:08.558+0000 7fde72fe5640 20 reqs_thread_entry: start 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -33> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command sync trace show hook 0x7fdfb80047f0 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -32> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command sync trace history hook 0x7fdfb80047f0 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -31> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command sync trace active hook 0x7fdfb80047f0 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -30> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command sync trace active_short hook 0x7fdfb80047f0 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -29> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -28> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 Filter name: none 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -27> 2024-03-04T18:06:08.558+0000 7fde70fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -26> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -25> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.637 INFO:tasks.workunit.client.0.smithi047.stderr: -24> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -23> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -22> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -21> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 : miss 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -20> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -19> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -18> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -17> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -16> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -15> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -14> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -13> 2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -12> 2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -11> 2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -10> 2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28, ofs0, ns] 2024-03-04T18:06:08.638 INFO:tasks.workunit.client.0.smithi047.stderr: -9> 2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595124 cookie 93855323373376 notifier 6539 bl.length()=873 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: -8> 2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: -7> 2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: -6> 2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: -5> 2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_sources= new_sources= 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: -4> 2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential sources added=[] removed=[] 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: -3> 2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_dests= new_dests= 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: -2> 2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential dests added=[] removed=[] 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: -1> 2024-03-04T18:06:08.582+0000 7fdfbf82a980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fdfbf82a980 time 2024-03-04T18:06:08.583168+0000 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fdfc30ffd7b] 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fdfc30fff2f] 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x555c626ed281] 2024-03-04T18:06:08.639 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x555c626f9d5d] 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fdfc1f6ad90] 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 0> 2024-03-04T18:06:08.582+0000 7fdfbf82a980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7fdfbf82a980 thread_name:radosgw-admin 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fdfc1f83520] 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:06:08.640 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fdfc30ffdd5] 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fdfc30fff2f] 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x555c626ed281] 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x555c626f9d5d] 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fdfc1f6ad90] 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr:--- logging levels --- 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 none 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 lockdep 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 context 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 crush 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds 2024-03-04T18:06:08.641 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_balancer 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_locker 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log_expire 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_migrator 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 buffer 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 timer 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 filer 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 striper 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 objecter 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rados 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_mirror 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_replay 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_pwl 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 journaler 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objectcacher 2024-03-04T18:06:08.642 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 client 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 osd 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 optracker 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objclass 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 filestore 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 journal 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 ms 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mon 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 0/10 monc 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 paxos 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 tp 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 auth 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 crypto 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 finisher 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 reserver 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 heartbeatmap 2024-03-04T18:06:08.643 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 perfcounter 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 20/20 rgw 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_sync 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_datacache 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_access 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_flight 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 javaclient 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 asok 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 throttle 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 refs 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 compressor 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluestore 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluefs 2024-03-04T18:06:08.644 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 bdev 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 kstore 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 4/ 5 rocksdb 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 fuse 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 2/ 5 mgr 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mgrc 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 dpdk 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 eventtrace 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 prioritycache 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 test 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephsqlite 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_onode 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_odata 2024-03-04T18:06:08.645 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_omap 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_tm 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_t 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_epm 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_lba 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cache 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_journal 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_device 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_backref 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 alienstore 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mclock 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cyanstore 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 ceph_exporter 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 memstore 2024-03-04T18:06:08.646 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 trace 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: -2/-2 (syslog threshold) 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 99/99 (stderr threshold) 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fde70fe1640 / http_manager 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fde72fe5640 / http_manager 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fde73fe7640 / rgw_dt_lg_renew 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fde8effd640 / ms_dispatch 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdea0ff9640 / io_context_pool 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdea2ffd640 / ms_dispatch 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdea9e03640 / ms_dispatch 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdeab606640 / io_context_pool 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbcb41640 / msgr-worker-2 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbd342640 / msgr-worker-1 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbdb43640 / msgr-worker-0 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbe344640 / io_context_pool 2024-03-04T18:06:08.647 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbf82a980 / radosgw-admin 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr: max_recent 500 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr: max_new 1000 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr: log_file /var/log/ceph/ceph-client.admin.99976.log 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:--- end dump of recent events --- 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:--- begin dump of recent events --- 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr: -499> 2024-03-04T18:06:08.482+0000 7fdfbcb41640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr: -498> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr: -497> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr: -496> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: dump: 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.648 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: -495> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: -494> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: -493> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6527 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: -492> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: -491> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: get_monmap_and_config success 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: -490> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 4 set_mon_vals no callback set 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: -489> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: -488> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr: -487> 2024-03-04T18:06:08.482+0000 7fdeab606640 10 monclient: dump: 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.649 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr: -486> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: shutdown 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr: -485> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 asok(0x555c6507a130) unregister_commands rotate-key 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr: -484> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: build_initial_monmap 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr: -483> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 build_initial for_mkfs: 0 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr: -482> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: monmap: 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:06:08.650 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -481> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding auth protocol: cephx 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -480> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding auth protocol: cephx 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -479> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding auth protocol: cephx 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -478> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -477> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -476> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -475> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -474> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -473> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -472> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.651 INFO:tasks.workunit.client.0.smithi047.stderr: -471> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -470> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -469> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -468> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: crc 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -467> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65267590) adding con mode: secure 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -466> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -465> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 librados: starting msgr at 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -464> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 librados: starting objecter 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -463> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command objecter_requests hook 0x555c6508f130 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -462> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 librados: setting wanted keys 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -461> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 1 librados: calling monclient init 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -460> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 10 monclient: init 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -459> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding auth protocol: cephx 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -458> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding auth protocol: cephx 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -457> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding auth protocol: cephx 2024-03-04T18:06:08.652 INFO:tasks.workunit.client.0.smithi047.stderr: -456> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -455> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -454> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -453> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -452> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -451> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -450> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -449> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -448> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -447> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -446> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: crc 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -445> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 5 AuthRegistry(0x555c65261d00) adding con mode: secure 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -444> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -443> 2024-03-04T18:06:08.482+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -442> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command rotate-key hook 0x555c65261e48 2024-03-04T18:06:08.653 INFO:tasks.workunit.client.0.smithi047.stderr: -441> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _reopen_session rank -1 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -440> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -439> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-b con 0x555c651af120 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -438> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-a con 0x555c651afdc0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -437> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-c con 0x555c65290bf0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -436> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -435> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -434> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -433> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): _renew_subs 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -432> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient(hunting): authenticate will time out at 2539.660087s 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -431> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request con 0x555c651afdc0 auth_method 0 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -430> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -429> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -428> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -427> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request con 0x555c65290bf0 auth_method 0 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -426> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.654 INFO:tasks.workunit.client.0.smithi047.stderr: -425> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -424> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -423> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -422> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -421> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -420> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -419> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -418> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -417> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient(hunting): get_auth_request con 0x555c651af120 auth_method 0 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -416> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -415> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -414> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -413> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_done global_id 6530 payload 824 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -412> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient: _finish_hunting 0 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -411> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 1 monclient: found mon.noname-c 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -410> 2024-03-04T18:06:08.486+0000 7fdfbdb43640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.655 INFO:tasks.workunit.client.0.smithi047.stderr: -409> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr: -408> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr: -407> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: dump: 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr: -406> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr: -405> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.656 INFO:tasks.workunit.client.0.smithi047.stderr: -404> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr: -403> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr: -402> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr: -401> 2024-03-04T18:06:08.486+0000 7fdea9e03640 10 monclient: dump: 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr: -400> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6530 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr: -399> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.657 INFO:tasks.workunit.client.0.smithi047.stderr: -398> 2024-03-04T18:06:08.486+0000 7fdeab606640 4 set_mon_vals no callback set 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -397> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -396> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -395> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -394> 2024-03-04T18:06:08.486+0000 7fdfbf82a980 1 librados: init done 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -393> 2024-03-04T18:06:08.486+0000 7fdea9e03640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -392> 2024-03-04T18:06:08.486+0000 7fdea9e03640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -391> 2024-03-04T18:06:08.486+0000 7fdea9e03640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -390> 2024-03-04T18:06:08.486+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fdea4065940 auth_method 0 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -389> 2024-03-04T18:06:08.486+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x555c65265ce0 auth_method 0 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -388> 2024-03-04T18:06:08.490+0000 7fdfbd342640 10 monclient: get_auth_request con 0x555c652ac910 auth_method 0 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -387> 2024-03-04T18:06:08.490+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x555c652b19a0 auth_method 0 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -386> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 10 monclient: build_initial_monmap 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -385> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 1 build_initial for_mkfs: 0 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr: -384> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 10 monclient: monmap: 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:06:08.658 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr: -383> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding auth protocol: cephx 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr: -382> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding auth protocol: cephx 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr: -381> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding auth protocol: cephx 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr: -380> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr: -379> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr: -378> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr: -377> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.659 INFO:tasks.workunit.client.0.smithi047.stderr: -376> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -375> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -374> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -373> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -372> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -371> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -370> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: crc 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -369> 2024-03-04T18:06:08.490+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b58c0) adding con mode: secure 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -368> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -367> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command objecter_requests cmddesc objecter_requests hook 0x555c652b9560 EEXIST 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -366> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: init 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -365> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding auth protocol: cephx 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -364> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding auth protocol: cephx 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -363> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding auth protocol: cephx 2024-03-04T18:06:08.660 INFO:tasks.workunit.client.0.smithi047.stderr: -362> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -361> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -360> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -359> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -358> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -357> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -356> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -355> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -354> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -353> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -352> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: crc 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -351> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 AuthRegistry(0x555c652b4948) adding con mode: secure 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -350> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -349> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -348> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command rotate-key cmddesc rotate-key hook 0x555c652b4a90 EEXIST 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -347> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _reopen_session rank -1 2024-03-04T18:06:08.661 INFO:tasks.workunit.client.0.smithi047.stderr: -346> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -345> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-b con 0x555c652dfcd0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -344> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-c con 0x555c652e00b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -343> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-a con 0x555c652db270 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -342> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -341> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -340> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -339> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): _renew_subs 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -338> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient(hunting): authenticate will time out at 2539.669000s 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -337> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request con 0x555c652db270 auth_method 0 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -336> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -335> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -334> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -333> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request con 0x555c652e00b0 auth_method 0 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -332> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -331> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.662 INFO:tasks.workunit.client.0.smithi047.stderr: -330> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -329> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -328> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -327> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -326> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -325> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -324> 2024-03-04T18:06:08.494+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -323> 2024-03-04T18:06:08.494+0000 7fdfbd342640 10 monclient(hunting): get_auth_request con 0x555c652dfcd0 auth_method 0 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -322> 2024-03-04T18:06:08.494+0000 7fdfbd342640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -321> 2024-03-04T18:06:08.494+0000 7fdfbd342640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -320> 2024-03-04T18:06:08.494+0000 7fdfbd342640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -319> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_done global_id 6954 payload 824 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -318> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient: _finish_hunting 0 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -317> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 1 monclient: found mon.noname-a 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -316> 2024-03-04T18:06:08.494+0000 7fdfbdb43640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -315> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi047.stderr: -314> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr: -313> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: dump: 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr: -312> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr: -311> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr: -310> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.664 INFO:tasks.workunit.client.0.smithi047.stderr: -309> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr: -308> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr: -307> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 10 monclient: dump: 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr: -306> 2024-03-04T18:06:08.494+0000 7fdfbe344640 4 set_mon_vals no callback set 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr: -305> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6954 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr: -304> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.665 INFO:tasks.workunit.client.0.smithi047.stderr: -303> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -302> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -301> 2024-03-04T18:06:08.494+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -300> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -299> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -298> 2024-03-04T18:06:08.494+0000 7fdea2ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -297> 2024-03-04T18:06:08.498+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fde98065880 auth_method 0 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -296> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: get_monmap_and_config 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -295> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: build_initial_monmap 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -294> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 build_initial for_mkfs: 0 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr: -293> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: monmap: 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:06:08.666 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -292> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -291> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -290> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -289> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -288> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -287> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -286> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -285> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -284> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.667 INFO:tasks.workunit.client.0.smithi047.stderr: -283> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -282> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -281> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -280> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -279> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -278> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -277> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -276> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: init 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -275> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding auth protocol: cephx 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -274> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding auth protocol: cephx 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -273> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding auth protocol: cephx 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -272> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -271> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -270> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.668 INFO:tasks.workunit.client.0.smithi047.stderr: -269> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -268> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -267> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -266> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -265> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -264> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -263> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -262> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: crc 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -261> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2570) adding con mode: secure 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -260> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -259> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -258> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command rotate-key cmddesc rotate-key hook 0x7ffea8ca26b8 EEXIST 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -257> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: _reopen_session rank -1 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -256> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -255> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-b con 0x555c65331df0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -254> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-a con 0x555c652ffc10 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:06:08.669 INFO:tasks.workunit.client.0.smithi047.stderr: -253> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-c con 0x555c6532d7b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -252> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -251> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -250> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -249> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): _renew_subs 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -248> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): authenticate will time out at 2539.673413s 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -247> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request con 0x555c652ffc10 auth_method 0 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -246> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -245> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -244> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -243> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request con 0x555c6532d7b0 auth_method 0 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -242> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -241> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -240> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -239> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.670 INFO:tasks.workunit.client.0.smithi047.stderr: -238> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -237> 2024-03-04T18:06:08.498+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -236> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -235> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -234> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -233> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient(hunting): handle_auth_done global_id 6536 payload 274 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -232> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient: _finish_hunting 0 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -231> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 1 monclient: found mon.noname-c 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -230> 2024-03-04T18:06:08.498+0000 7fdfbdb43640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -229> 2024-03-04T18:06:08.498+0000 7fdfbd342640 10 monclient: get_auth_request con 0x555c65331df0 auth_method 0 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -228> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -227> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr: -226> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: dump: 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.671 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr: -225> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr: -224> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr: -223> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6536 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr: -222> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr: -221> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: get_monmap_and_config success 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr: -220> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 4 set_mon_vals no callback set 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr: -219> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.672 INFO:tasks.workunit.client.0.smithi047.stderr: -218> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr: -217> 2024-03-04T18:06:08.498+0000 7fdea0ff9640 10 monclient: dump: 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr: -216> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: shutdown 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr: -215> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: build_initial_monmap 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr: -214> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 build_initial for_mkfs: 0 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr: -213> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: monmap: 2024-03-04T18:06:08.673 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 0 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 0.000000 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:created 0.000000 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 0 (unknown) 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.noname-b 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-c 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr: -212> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr: -211> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr: -210> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding auth protocol: cephx 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr: -209> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr: -208> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr: -207> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.674 INFO:tasks.workunit.client.0.smithi047.stderr: -206> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -205> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -204> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -203> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -202> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -201> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -200> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -199> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: crc 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -198> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fd820) adding con mode: secure 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -197> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -196> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 librados: starting msgr at 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -195> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 librados: starting objecter 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -194> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command objecter_requests cmddesc objecter_requests hook 0x555c65328490 EEXIST 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -193> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 librados: setting wanted keys 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -192> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 1 librados: calling monclient init 2024-03-04T18:06:08.675 INFO:tasks.workunit.client.0.smithi047.stderr: -191> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: init 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -190> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding auth protocol: cephx 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -189> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding auth protocol: cephx 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -188> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding auth protocol: cephx 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -187> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -186> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -185> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -184> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -183> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -182> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -181> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -180> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -179> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -178> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -177> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: crc 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -176> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 AuthRegistry(0x555c652fc9f0) adding con mode: secure 2024-03-04T18:06:08.676 INFO:tasks.workunit.client.0.smithi047.stderr: -175> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -174> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -173> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command rotate-key cmddesc rotate-key hook 0x555c652fcb38 EEXIST 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -172> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: _reopen_session rank -1 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -171> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -170> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-a con 0x555c652ffc10 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -169> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-c con 0x555c6532d7b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -168> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): picked mon.noname-b con 0x555c65331df0 addr [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -167> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -166> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -165> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -164> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): _renew_subs 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -163> 2024-03-04T18:06:08.498+0000 7fdfbf82a980 10 monclient(hunting): authenticate will time out at 2539.675999s 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -162> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request con 0x555c6532d7b0 auth_method 0 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -161> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -160> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.677 INFO:tasks.workunit.client.0.smithi047.stderr: -159> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -158> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): get_auth_request con 0x555c652ffc10 auth_method 0 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -157> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -156> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -155> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -154> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -153> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -152> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -151> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -150> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -149> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -148> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient(hunting): handle_auth_done global_id 6539 payload 824 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -147> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient: _finish_hunting 0 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -146> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 1 monclient: found mon.noname-c 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -145> 2024-03-04T18:06:08.502+0000 7fdfbcb41640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -144> 2024-03-04T18:06:08.502+0000 7fdfbdb43640 10 monclient: get_auth_request con 0x555c65331df0 auth_method 0 2024-03-04T18:06:08.678 INFO:tasks.workunit.client.0.smithi047.stderr: -143> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr: -142> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr: -141> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: dump: 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr: -140> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: _finish_auth 0 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr: -139> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: _check_auth_tickets 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr: -138> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 5 monclient: authenticate success, global_id 6539 2024-03-04T18:06:08.679 INFO:tasks.workunit.client.0.smithi047.stderr: -137> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr: -136> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr: -135> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr: -134> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr: -133> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr: -132> 2024-03-04T18:06:08.502+0000 7fde8effd640 10 monclient: dump: 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr:epoch 1 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr:fsid 254cdce0-6dc3-4167-bfd3-d6bd440b629e 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr:last_changed 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr:created 2024-03-04T17:40:51.199624+0000 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr:min_mon_release 19 (squid) 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr:election_strategy: 1 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-03-04T18:06:08.680 INFO:tasks.workunit.client.0.smithi047.stderr:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.c 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -131> 2024-03-04T18:06:08.502+0000 7fdea0ff9640 4 set_mon_vals no callback set 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -130> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 10 monclient: _renew_subs 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -129> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -128> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 1 librados: init done 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -127> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cr dump hook 0x555c65322af8 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -126> 2024-03-04T18:06:08.502+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -125> 2024-03-04T18:06:08.502+0000 7fde8effd640 4 mgrc handle_mgr_map Got map version 5 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -124> 2024-03-04T18:06:08.502+0000 7fde8effd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -123> 2024-03-04T18:06:08.502+0000 7fde8effd640 4 mgrc reconnect Starting new session with [v2:172.21.15.47:6832/724686481,v1:172.21.15.47:6833/724686481] 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -122> 2024-03-04T18:06:08.502+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fde88065940 auth_method 0 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -121> 2024-03-04T18:06:08.502+0000 7fdfbdb43640 10 monclient: get_auth_request con 0x555c6534b3b0 auth_method 0 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -120> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -119> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 realm 2024-03-04T18:06:08.681 INFO:tasks.workunit.client.0.smithi047.stderr: -118> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -117> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -116> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -115> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -114> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -113> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -112> 2024-03-04T18:06:08.506+0000 7fdfbd342640 10 monclient: get_auth_request con 0x555c6534f620 auth_method 0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -111> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -110> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -109> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -108> 2024-03-04T18:06:08.506+0000 7fdfbf82a980 20 searching for the correct realm 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -107> 2024-03-04T18:06:08.506+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x555c65354730 auth_method 0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -106> 2024-03-04T18:06:08.510+0000 7fdfbdb43640 10 monclient: get_auth_request con 0x7fdfb4008070 auth_method 0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -105> 2024-03-04T18:06:08.510+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fdfb0008040 auth_method 0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -104> 2024-03-04T18:06:08.514+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x7fdfb40659d0 auth_method 0 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -103> 2024-03-04T18:06:08.514+0000 7fdfbcb41640 0 -- 172.21.15.47:0/3463461626 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fdfb40659d0 msgr2=0x7fdfb408b820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.682 INFO:tasks.workunit.client.0.smithi047.stderr: -102> 2024-03-04T18:06:08.514+0000 7fde8effd640 1 client.6539.objecter ms_handle_reset 0x7fdfb40659d0 session 0x7fdfb4003b20 osd.5 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -101> 2024-03-04T18:06:08.514+0000 7fde8effd640 10 monclient: _renew_subs 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -100> 2024-03-04T18:06:08.514+0000 7fde8effd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3301/0 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -99> 2024-03-04T18:06:08.514+0000 7fdfbcb41640 10 monclient: get_auth_request con 0x7fde88075be0 auth_method 0 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -98> 2024-03-04T18:06:08.518+0000 7fdfbdb43640 10 monclient: get_auth_request con 0x7fdfb0051040 auth_method 0 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -97> 2024-03-04T18:06:08.518+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fdfb8124c20 auth_method 0 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -96> 2024-03-04T18:06:08.518+0000 7fdfbd342640 0 -- 172.21.15.47:0/3463461626 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fdfb8124c20 msgr2=0x7fdfb80f7b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -95> 2024-03-04T18:06:08.518+0000 7fde8effd640 1 client.6539.objecter ms_handle_reset 0x7fdfb8124c20 session 0x7fdfb8107700 osd.3 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -94> 2024-03-04T18:06:08.522+0000 7fdfbd342640 10 monclient: get_auth_request con 0x7fdfb40659d0 auth_method 0 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -93> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -92> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -91> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -90> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -89> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -88> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -87> 2024-03-04T18:06:08.526+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.683 INFO:tasks.workunit.client.0.smithi047.stderr: -86> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -85> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -84> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -83> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -82> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -81> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 zone default found 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -80> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 4 Realm: () 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -79> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -78> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -77> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -76> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 zonegroup default 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -75> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -74> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -73> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.684 INFO:tasks.workunit.client.0.smithi047.stderr: -72> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -71> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -70> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -69> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 started sync module instance, tier type = 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -68> 2024-03-04T18:06:08.530+0000 7fdfbf82a980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -67> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=0 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -66> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=1 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -65> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=2 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -64> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=3 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -63> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=4 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -62> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=5 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -61> 2024-03-04T18:06:08.550+0000 7fdfbf82a980 20 add_watcher() i=6 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -60> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 20 add_watcher() i=7 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -59> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:08.685 INFO:tasks.workunit.client.0.smithi047.stderr: -58> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding auth protocol: cephx 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -57> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding auth protocol: cephx 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -56> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding auth protocol: cephx 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -55> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -54> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -53> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -52> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -51> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -50> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -49> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -48> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -47> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -46> 2024-03-04T18:06:08.554+0000 7fde73fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -45> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.686 INFO:tasks.workunit.client.0.smithi047.stderr: -44> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: crc 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -43> 2024-03-04T18:06:08.554+0000 7fdfbf82a980 5 AuthRegistry(0x7ffea8ca2a00) adding con mode: secure 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -42> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -41> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -40> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -39> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cache list hook 0x555c65329c20 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -38> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cache inspect hook 0x555c65329c20 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -37> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cache erase hook 0x555c65329c20 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -36> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command cache zap hook 0x555c65329c20 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -35> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 note: GC not initialized 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -34> 2024-03-04T18:06:08.558+0000 7fde72fe5640 20 reqs_thread_entry: start 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -33> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command sync trace show hook 0x7fdfb80047f0 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -32> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command sync trace history hook 0x7fdfb80047f0 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -31> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command sync trace active hook 0x7fdfb80047f0 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -30> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 5 asok(0x555c6507a130) register_command sync trace active_short hook 0x7fdfb80047f0 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -29> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -28> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 Filter name: none 2024-03-04T18:06:08.687 INFO:tasks.workunit.client.0.smithi047.stderr: -27> 2024-03-04T18:06:08.558+0000 7fde70fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -26> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -25> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -24> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -23> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -22> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -21> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 : miss 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -20> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -19> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -18> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -17> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -16> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -15> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -14> 2024-03-04T18:06:08.558+0000 7fdfbf82a980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 2024-03-04T18:06:08.688 INFO:tasks.workunit.client.0.smithi047.stderr: -13> 2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -12> 2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -11> 2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -10> 2024-03-04T18:06:08.570+0000 7fdfbf82a980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28, ofs0, ns] 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -9> 2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595124 cookie 93855323373376 notifier 6539 bl.length()=873 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -8> 2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -7> 2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -6> 2024-03-04T18:06:08.570+0000 7fdea0ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -5> 2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_sources= new_sources= 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -4> 2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential sources added=[] removed=[] 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -3> 2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_dests= new_dests= 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -2> 2024-03-04T18:06:08.574+0000 7fdfbf82a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential dests added=[] removed=[] 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: -1> 2024-03-04T18:06:08.582+0000 7fdfbf82a980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fdfbf82a980 time 2024-03-04T18:06:08.583168+0000 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:06:08.689 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fdfc30ffd7b] 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fdfc30fff2f] 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x555c626ed281] 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x555c626f9d5d] 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 5: main() 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fdfc1f6ad90] 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 7: __libc_start_main() 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 8: _start() 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 0> 2024-03-04T18:06:08.582+0000 7fdfbf82a980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: in thread 7fdfbf82a980 thread_name:radosgw-admin 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:06:08.690 INFO:tasks.workunit.client.0.smithi047.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fdfc1f83520] 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 2: pthread_kill() 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 3: raise() 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 4: abort() 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fdfc30ffdd5] 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fdfc30fff2f] 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x555c626ed281] 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x555c626f9d5d] 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 9: main() 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fdfc1f6ad90] 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 11: __libc_start_main() 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 12: _start() 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:08.691 INFO:tasks.workunit.client.0.smithi047.stderr:--- logging levels --- 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 none 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 lockdep 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 context 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 crush 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_balancer 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_locker 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_log_expire 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mds_migrator 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 buffer 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 timer 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 filer 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 striper 2024-03-04T18:06:08.692 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 1 objecter 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rados 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_mirror 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_replay 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 rbd_pwl 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 journaler 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objectcacher 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 client 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 osd 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 optracker 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 objclass 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 filestore 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 journal 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 ms 2024-03-04T18:06:08.693 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mon 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 0/10 monc 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 paxos 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 tp 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 auth 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 crypto 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 finisher 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 reserver 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 heartbeatmap 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 perfcounter 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 20/20 rgw 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_sync 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_datacache 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_access 2024-03-04T18:06:08.694 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_flight 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 javaclient 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 asok 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 1 throttle 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 0 refs 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 compressor 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluestore 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 bluefs 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 3 bdev 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 kstore 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 4/ 5 rocksdb 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 fuse 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 2/ 5 mgr 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mgrc 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 dpdk 2024-03-04T18:06:08.695 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 eventtrace 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 prioritycache 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 test 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cephsqlite 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_onode 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_odata 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_omap 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_tm 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_t 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_epm 2024-03-04T18:06:08.696 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_lba 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_cache 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_journal 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_device 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 seastore_backref 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 alienstore 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 mclock 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 0/ 5 cyanstore 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 ceph_exporter 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 memstore 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 1/ 5 trace 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: -2/-2 (syslog threshold) 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 99/99 (stderr threshold) 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 7fde70fe1640 / http_manager 2024-03-04T18:06:08.697 INFO:tasks.workunit.client.0.smithi047.stderr: 7fde72fe5640 / http_manager 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fde73fe7640 / rgw_dt_lg_renew 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fde8effd640 / ms_dispatch 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdea0ff9640 / io_context_pool 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdea2ffd640 / ms_dispatch 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdea9e03640 / ms_dispatch 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdeab606640 / io_context_pool 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbcb41640 / msgr-worker-2 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbd342640 / msgr-worker-1 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbdb43640 / msgr-worker-0 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbe344640 / io_context_pool 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: 7fdfbf82a980 / radosgw-admin 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: max_recent 500 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: max_new 1000 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr: log_file /var/lib/ceph/crash/2024-03-04T18:06:08.585138Z_edfd671a-e57c-4cca-858f-23843fe13fd9/log 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr:--- end dump of recent events --- 2024-03-04T18:06:08.698 INFO:tasks.workunit.client.0.smithi047.stderr:Aborted 2024-03-04T18:06:08.699 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.662+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.662+0000 7fe94893e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.662+0000 7fe94893e980 20 realm 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.662+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.662+0000 7fe94893e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.662+0000 7fe94893e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.662+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.666+0000 7fe94893e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.666+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.666+0000 7fe94893e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.666+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.666+0000 7fe94893e980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.666+0000 7fe94893e980 20 searching for the correct realm 2024-03-04T18:06:08.701 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.670+0000 7fe946456640 0 -- 172.21.15.47:0/1087949229 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fe938043060 msgr2=0x7fe938033140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.879 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:08.882+0000 7fe946456640 0 -- 172.21.15.47:0/1087949229 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fe930003ac0 msgr2=0x7fe9400079e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:09.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.098+0000 7fe94893e980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.098+0000 7fe94893e980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.098+0000 7fe94893e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.098+0000 7fe94893e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.098+0000 7fe94893e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.098+0000 7fe94893e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.098+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:09.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 zone default found 2024-03-04T18:06:09.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 4 Realm: () 2024-03-04T18:06:09.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:09.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:09.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:09.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 zonegroup default 2024-03-04T18:06:09.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.102+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.106+0000 7fe94893e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.106+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.106+0000 7fe94893e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.106+0000 7fe94893e980 20 started sync module instance, tier type = 2024-03-04T18:06:09.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.106+0000 7fe94893e980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:09.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.130+0000 7fe94893e980 20 add_watcher() i=0 2024-03-04T18:06:09.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.130+0000 7fe94893e980 20 add_watcher() i=1 2024-03-04T18:06:09.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.130+0000 7fe94893e980 20 add_watcher() i=2 2024-03-04T18:06:09.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.130+0000 7fe94893e980 20 add_watcher() i=3 2024-03-04T18:06:09.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.130+0000 7fe94893e980 20 add_watcher() i=4 2024-03-04T18:06:09.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.130+0000 7fe94893e980 20 add_watcher() i=5 2024-03-04T18:06:09.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.130+0000 7fe94893e980 20 add_watcher() i=6 2024-03-04T18:06:09.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.130+0000 7fe94893e980 20 add_watcher() i=7 2024-03-04T18:06:09.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.130+0000 7fe94893e980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:09.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.134+0000 7fe7fcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:09.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe94893e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:09.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe94893e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:09.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe94893e980 5 note: GC not initialized 2024-03-04T18:06:09.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe7777fe640 20 reqs_thread_entry: start 2024-03-04T18:06:09.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe94893e980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:09.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe94893e980 20 Filter name: none 2024-03-04T18:06:09.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe7757fa640 20 reqs_thread_entry: start 2024-03-04T18:06:09.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe94893e980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:09.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.138+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.142+0000 7fe94893e980 20 rados_obj.operate() r=0 bl.length=403 2024-03-04T18:06:09.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.142+0000 7fe94893e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:09.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.142+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.142+0000 7fe94893e980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:09.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.142+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.142+0000 7fe94893e980 20 rados_obj.operate() r=0 bl.length=403 2024-03-04T18:06:09.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.142+0000 7fe94893e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:09.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.146+0000 7fe94893e980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.146+0000 7fe94893e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.146+0000 7fe7fcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:09.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.150+0000 7fe94893e980 20 remove_watcher() i=0 2024-03-04T18:06:09.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.150+0000 7fe94893e980 2 removed watcher, disabling cache 2024-03-04T18:06:09.149 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.154+0000 7fe94893e980 20 remove_watcher() i=1 2024-03-04T18:06:09.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.154+0000 7fe94893e980 20 remove_watcher() i=2 2024-03-04T18:06:09.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.158+0000 7fe94893e980 20 remove_watcher() i=3 2024-03-04T18:06:09.155 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.158+0000 7fe94893e980 20 remove_watcher() i=4 2024-03-04T18:06:09.157 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.162+0000 7fe94893e980 20 remove_watcher() i=5 2024-03-04T18:06:09.160 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.166+0000 7fe94893e980 20 remove_watcher() i=6 2024-03-04T18:06:09.163 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:09.166+0000 7fe94893e980 20 remove_watcher() i=7 2024-03-04T18:06:09.179 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:09.179 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:09.179 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-do-reshard id: 797b67a4-b876-4d04-b122-10afd858d465.4453.28 num_objects: 20 size_kb: 1 num_shards: 11 2024-03-04T18:06:09.179 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:09.181 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:09.181 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:39.419 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:39.419 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:39.506 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:06:39.544 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:39.544 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:39.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.593+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:39.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.593+0000 7fb08d3b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:39.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.593+0000 7fb08d3b2980 20 realm 2024-03-04T18:06:39.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.593+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:39.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.593+0000 7fb08d3b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:39.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.593+0000 7fb08d3b2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:39.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.593+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:39.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.593+0000 7fb08d3b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:39.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.593+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:39.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.597+0000 7fb08d3b2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:39.593 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.597+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:39.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.597+0000 7fb08d3b2980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:39.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.597+0000 7fb08d3b2980 20 searching for the correct realm 2024-03-04T18:06:39.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.601+0000 7fb08aeca640 0 -- 172.21.15.47:0/1068832925 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fb0800ba070 msgr2=0x7fb08010d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:39.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:39.813+0000 7fb08aeca640 0 -- 172.21.15.47:0/1068832925 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fb084031040 msgr2=0x7fb084031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:40.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:40.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:40.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:40.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:40.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:40.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:40.030 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:40.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:40.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 zone default found 2024-03-04T18:06:40.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 4 Realm: () 2024-03-04T18:06:40.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:40.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:40.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:40.032 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 zonegroup default 2024-03-04T18:06:40.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.033+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.037+0000 7fb08d3b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.037+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.037+0000 7fb08d3b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.033 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.037+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.037+0000 7fb08d3b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.037+0000 7fb08d3b2980 20 started sync module instance, tier type = 2024-03-04T18:06:40.034 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.037+0000 7fb08d3b2980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:40.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.061+0000 7fb08d3b2980 20 add_watcher() i=0 2024-03-04T18:06:40.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.061+0000 7fb08d3b2980 20 add_watcher() i=1 2024-03-04T18:06:40.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.061+0000 7fb08d3b2980 20 add_watcher() i=2 2024-03-04T18:06:40.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.061+0000 7fb08d3b2980 20 add_watcher() i=3 2024-03-04T18:06:40.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.061+0000 7fb08d3b2980 20 add_watcher() i=4 2024-03-04T18:06:40.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.061+0000 7fb08d3b2980 20 add_watcher() i=5 2024-03-04T18:06:40.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.061+0000 7fb08d3b2980 20 add_watcher() i=6 2024-03-04T18:06:40.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.061+0000 7fb08d3b2980 20 add_watcher() i=7 2024-03-04T18:06:40.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.061+0000 7fb08d3b2980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:40.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.065+0000 7faf417e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:40.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:40.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:40.065 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 5 note: GC not initialized 2024-03-04T18:06:40.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7faebbfff640 20 reqs_thread_entry: start 2024-03-04T18:06:40.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:40.066 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 20 Filter name: none 2024-03-04T18:06:40.067 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7faeb9ffb640 20 reqs_thread_entry: start 2024-03-04T18:06:40.067 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-03-04T18:06:40.067 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.067 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:40.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-03-04T18:06:40.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-03-04T18:06:40.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 : miss 2024-03-04T18:06:40.068 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.069+0000 7fb08d3b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.073+0000 7fb08d3b2980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:40.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.073+0000 7fb08d3b2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:40.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.073+0000 7fb08d3b2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:40.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.073+0000 7fb08d3b2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:40.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.073+0000 7fb08d3b2980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:40.069 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.073+0000 7fb08d3b2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 2024-03-04T18:06:40.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7fb08d3b2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:40.080 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7fb08d3b2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:40.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7fb08d3b2980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:40.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7fb08d3b2980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28, ofs0, ns] 2024-03-04T18:06:40.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7faf6dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595126 cookie 94462140798912 notifier 6569 bl.length()=873 2024-03-04T18:06:40.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7faf6dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:40.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7faf6dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:40.081 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7faf6dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:40.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7fb08d3b2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_sources= new_sources= 2024-03-04T18:06:40.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7fb08d3b2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential sources added=[] removed=[] 2024-03-04T18:06:40.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7fb08d3b2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_dests= new_dests= 2024-03-04T18:06:40.083 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.085+0000 7fb08d3b2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential dests added=[] removed=[] 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.0 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.0 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.1 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.1 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.2 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.2 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.3 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.3 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.4 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.4 2024-03-04T18:06:40.089 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.5 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.5 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.6 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.6 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.7 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.7 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.8 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.8 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.9 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.9 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.10 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.10 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.11 2024-03-04T18:06:40.090 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1.11 2024-03-04T18:06:40.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.0 2024-03-04T18:06:40.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.1 2024-03-04T18:06:40.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.2 2024-03-04T18:06:40.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.093+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.3 2024-03-04T18:06:40.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.097+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.4 2024-03-04T18:06:40.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.097+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.5 2024-03-04T18:06:40.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.097+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.6 2024-03-04T18:06:40.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.097+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.7 2024-03-04T18:06:40.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.097+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.8 2024-03-04T18:06:40.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.101+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.9 2024-03-04T18:06:40.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.101+0000 7fb08d3b2980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.28.10 2024-03-04T18:06:40.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.109+0000 7fb08d3b2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:40.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.109+0000 7fb08d3b2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 2024-03-04T18:06:40.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.113+0000 7fb08d3b2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:40.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.113+0000 7fb08d3b2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:40.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.113+0000 7fb08d3b2980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:40.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.113+0000 7fb08d3b2980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28, ofs0, ns] 2024-03-04T18:06:40.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.113+0000 7faf6dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595127 cookie 94462140798912 notifier 6569 bl.length()=841 2024-03-04T18:06:40.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.113+0000 7faf6dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 info.flags=0x17 2024-03-04T18:06:40.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.113+0000 7faf6dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:797b67a4-b876-4d04-b122-10afd858d465.4453.28 to cache LRU end 2024-03-04T18:06:40.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.113+0000 7faf6dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-03-04T18:06:40.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.117+0000 7fb08d3b2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_sources= new_sources= 2024-03-04T18:06:40.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.117+0000 7fb08d3b2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential sources added=[] removed=[] 2024-03-04T18:06:40.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.117+0000 7fb08d3b2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): orig_dests= new_dests= 2024-03-04T18:06:40.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.117+0000 7fb08d3b2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[797b67a4-b876-4d04-b122-10afd858d465.4453.28]): potential dests added=[] removed=[] 2024-03-04T18:06:40.118 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.121+0000 7fb08d3b2980 1 execute INFO: reshard of bucket "abort-at-do-reshard" completed successfully 2024-03-04T18:06:40.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.125+0000 7faf417e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:40.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.129+0000 7fb08d3b2980 20 remove_watcher() i=0 2024-03-04T18:06:40.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.129+0000 7fb08d3b2980 2 removed watcher, disabling cache 2024-03-04T18:06:40.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.129+0000 7fb08d3b2980 20 remove_watcher() i=1 2024-03-04T18:06:40.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.133+0000 7fb08d3b2980 20 remove_watcher() i=2 2024-03-04T18:06:40.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.133+0000 7fb08d3b2980 20 remove_watcher() i=3 2024-03-04T18:06:40.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.137+0000 7fb08d3b2980 20 remove_watcher() i=4 2024-03-04T18:06:40.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.137+0000 7fb08d3b2980 20 remove_watcher() i=5 2024-03-04T18:06:40.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.141+0000 7fb08d3b2980 20 remove_watcher() i=6 2024-03-04T18:06:40.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.145+0000 7fb08d3b2980 20 remove_watcher() i=7 2024-03-04T18:06:40.159 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-03-04T18:06:40.184 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:40.184 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:40.227 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.229+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 20 realm 2024-03-04T18:06:40.229 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:40.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.231 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:40.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.233+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.237+0000 7f910a13a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:40.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.237+0000 7f910a13a980 20 searching for the correct realm 2024-03-04T18:06:40.239 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.241+0000 7f9107451640 0 -- 172.21.15.47:0/1842446104 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f90fc0dc920 msgr2=0x7f90fc0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:40.453 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.457+0000 7f9107c52640 0 -- 172.21.15.47:0/1842446104 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f910001d070 msgr2=0x7f910007a390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:40.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.469+0000 7f910a13a980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:40.467 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.469+0000 7f910a13a980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:40.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.469+0000 7f910a13a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:40.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.469+0000 7f910a13a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:40.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.469+0000 7f910a13a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:40.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.469+0000 7f910a13a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:40.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.469+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.469+0000 7f910a13a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.468 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.469+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.469 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:40.469 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:40.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 zone default found 2024-03-04T18:06:40.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 4 Realm: () 2024-03-04T18:06:40.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:40.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:40.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:40.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 zonegroup default 2024-03-04T18:06:40.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.471 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 started sync module instance, tier type = 2024-03-04T18:06:40.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.473+0000 7f910a13a980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:40.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.497+0000 7f910a13a980 20 add_watcher() i=0 2024-03-04T18:06:40.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.497+0000 7f910a13a980 20 add_watcher() i=1 2024-03-04T18:06:40.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.497+0000 7f910a13a980 20 add_watcher() i=2 2024-03-04T18:06:40.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.497+0000 7f910a13a980 20 add_watcher() i=3 2024-03-04T18:06:40.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.497+0000 7f910a13a980 20 add_watcher() i=4 2024-03-04T18:06:40.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.497+0000 7f910a13a980 20 add_watcher() i=5 2024-03-04T18:06:40.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.497+0000 7f910a13a980 20 add_watcher() i=6 2024-03-04T18:06:40.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.501+0000 7f910a13a980 20 add_watcher() i=7 2024-03-04T18:06:40.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.501+0000 7f910a13a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:40.501 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f8fbdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:40.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f910a13a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:40.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f910a13a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:40.502 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f910a13a980 5 note: GC not initialized 2024-03-04T18:06:40.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f8fbcfe1640 20 reqs_thread_entry: start 2024-03-04T18:06:40.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f910a13a980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:40.503 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f910a13a980 20 Filter name: none 2024-03-04T18:06:40.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f8f367fc640 20 reqs_thread_entry: start 2024-03-04T18:06:40.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f910a13a980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:40.504 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.505+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.509+0000 7f910a13a980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:40.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.509+0000 7f910a13a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:40.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.509+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.506 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.509+0000 7f910a13a980 20 rados_obj.operate() r=0 bl.length=187 2024-03-04T18:06:40.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.509+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.509+0000 7f910a13a980 20 rados_obj.operate() r=0 bl.length=371 2024-03-04T18:06:40.507 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.509+0000 7f910a13a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:40.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.513+0000 7f910a13a980 20 rados->read ofs=0 len=0 2024-03-04T18:06:40.510 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.513+0000 7f910a13a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:40.512 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.513+0000 7f8fbdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:40.516 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.521+0000 7f910a13a980 20 remove_watcher() i=0 2024-03-04T18:06:40.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.521+0000 7f910a13a980 2 removed watcher, disabling cache 2024-03-04T18:06:40.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.521+0000 7f910a13a980 20 remove_watcher() i=1 2024-03-04T18:06:40.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.525+0000 7f910a13a980 20 remove_watcher() i=2 2024-03-04T18:06:40.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.525+0000 7f910a13a980 20 remove_watcher() i=3 2024-03-04T18:06:40.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.529+0000 7f910a13a980 20 remove_watcher() i=4 2024-03-04T18:06:40.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.533+0000 7f910a13a980 20 remove_watcher() i=5 2024-03-04T18:06:40.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.533+0000 7f910a13a980 20 remove_watcher() i=6 2024-03-04T18:06:40.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:40.537+0000 7f910a13a980 20 remove_watcher() i=7 2024-03-04T18:06:40.553 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:40.553 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:40.553 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: abort-at-do-reshard id: 797b67a4-b876-4d04-b122-10afd858d465.4453.28 num_objects: 19 size_kb: 1 num_shards: 12 2024-03-04T18:06:40.553 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:40.555 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:40.555 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:40.583 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:40.583 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:41.005 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:41.005 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:41.389 INFO:tasks.workunit.client.0.smithi047.stderr: test: reshard versioned bucket 2024-03-04T18:06:41.390 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket myver 2024-03-04T18:06:41.425 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:41.425 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:41.470 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.473+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.472 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 realm 2024-03-04T18:06:41.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.473 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:41.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.474 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:41.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:41.475 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.477+0000 7fa643550980 20 searching for the correct realm 2024-03-04T18:06:41.482 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.485+0000 7fa640867640 0 -- 172.21.15.47:0/122001209 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fa638048fb0 msgr2=0x7fa638084a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:41.489 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.493+0000 7fa641869640 0 -- 172.21.15.47:0/122001209 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55ad61808100 msgr2=0x55ad6180a570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:41.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:41.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:41.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:41.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:41.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:41.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:41.497 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:41.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:41.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 zone default found 2024-03-04T18:06:41.498 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 4 Realm: () 2024-03-04T18:06:41.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:41.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:41.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:41.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 zonegroup default 2024-03-04T18:06:41.499 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 started sync module instance, tier type = 2024-03-04T18:06:41.500 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.501+0000 7fa643550980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:41.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.521+0000 7fa643550980 20 add_watcher() i=0 2024-03-04T18:06:41.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.521+0000 7fa643550980 20 add_watcher() i=1 2024-03-04T18:06:41.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.521+0000 7fa643550980 20 add_watcher() i=2 2024-03-04T18:06:41.517 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.521+0000 7fa643550980 20 add_watcher() i=3 2024-03-04T18:06:41.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.521+0000 7fa643550980 20 add_watcher() i=4 2024-03-04T18:06:41.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.521+0000 7fa643550980 20 add_watcher() i=5 2024-03-04T18:06:41.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.521+0000 7fa643550980 20 add_watcher() i=6 2024-03-04T18:06:41.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.521+0000 7fa643550980 20 add_watcher() i=7 2024-03-04T18:06:41.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.521+0000 7fa643550980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:41.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.525+0000 7fa4f77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:41.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.525+0000 7fa643550980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:41.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.525+0000 7fa643550980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:41.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.525+0000 7fa643550980 5 note: GC not initialized 2024-03-04T18:06:41.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.525+0000 7fa4f67e4640 20 reqs_thread_entry: start 2024-03-04T18:06:41.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.525+0000 7fa643550980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:41.524 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.525+0000 7fa643550980 20 Filter name: none 2024-03-04T18:06:41.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.525+0000 7fa467fff640 20 reqs_thread_entry: start 2024-03-04T18:06:41.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.525+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:41.525 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:41.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:41.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:41.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:41.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:41.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.529+0000 7fa643550980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.533+0000 7fa643550980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.533+0000 7fa4f77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:41.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.537+0000 7fa643550980 20 remove_watcher() i=0 2024-03-04T18:06:41.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.537+0000 7fa643550980 2 removed watcher, disabling cache 2024-03-04T18:06:41.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.537+0000 7fa643550980 20 remove_watcher() i=1 2024-03-04T18:06:41.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.541+0000 7fa643550980 20 remove_watcher() i=2 2024-03-04T18:06:41.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.541+0000 7fa643550980 20 remove_watcher() i=3 2024-03-04T18:06:41.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.545+0000 7fa643550980 20 remove_watcher() i=4 2024-03-04T18:06:41.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.549+0000 7fa643550980 20 remove_watcher() i=5 2024-03-04T18:06:41.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.549+0000 7fa643550980 20 remove_watcher() i=6 2024-03-04T18:06:41.551 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.553+0000 7fa643550980 20 remove_watcher() i=7 2024-03-04T18:06:41.565 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:41.565 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:41.565 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: myver id: 797b67a4-b876-4d04-b122-10afd858d465.4453.18 num_objects: 0 size_kb: 0 num_shards: 11 2024-03-04T18:06:41.566 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:41.566 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 12 2024-03-04T18:06:41.594 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:41.595 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:41.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.641+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 realm 2024-03-04T18:06:41.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:41.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:41.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:41.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.645+0000 7f515c132980 20 searching for the correct realm 2024-03-04T18:06:41.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.653+0000 7f5159449640 0 -- 172.21.15.47:0/332347193 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f5150045770 msgr2=0x7f515007b930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:41.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.657+0000 7f5159c4a640 0 -- 172.21.15.47:0/332347193 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f515401e920 msgr2=0x7f51540383f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:41.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:41.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:41.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:41.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:41.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:41.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:41.665 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:41.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:41.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 zone default found 2024-03-04T18:06:41.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 4 Realm: () 2024-03-04T18:06:41.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:41.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:41.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:41.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 zonegroup default 2024-03-04T18:06:41.667 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:41.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 started sync module instance, tier type = 2024-03-04T18:06:41.668 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.669+0000 7f515c132980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:41.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.689+0000 7f515c132980 20 add_watcher() i=0 2024-03-04T18:06:41.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.689+0000 7f515c132980 20 add_watcher() i=1 2024-03-04T18:06:41.685 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.689+0000 7f515c132980 20 add_watcher() i=2 2024-03-04T18:06:41.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.689+0000 7f515c132980 20 add_watcher() i=3 2024-03-04T18:06:41.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.689+0000 7f515c132980 20 add_watcher() i=4 2024-03-04T18:06:41.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.689+0000 7f515c132980 20 add_watcher() i=5 2024-03-04T18:06:41.686 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.689+0000 7f515c132980 20 add_watcher() i=6 2024-03-04T18:06:41.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.689+0000 7f515c132980 20 add_watcher() i=7 2024-03-04T18:06:41.688 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.689+0000 7f515c132980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:41.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f500b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:41.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f515c132980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:41.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f515c132980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:41.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f515c132980 5 note: GC not initialized 2024-03-04T18:06:41.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f500a7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:41.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f515c132980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:41.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f515c132980 20 Filter name: none 2024-03-04T18:06:41.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f4f7bfff640 20 reqs_thread_entry: start 2024-03-04T18:06:41.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f515c132980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-03-04T18:06:41.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.693+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:41.693 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-03-04T18:06:41.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-03-04T18:06:41.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 : miss 2024-03-04T18:06:41.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 20 rados->read ofs=0 len=0 2024-03-04T18:06:41.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:41.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:41.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 10 adding default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:41.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:41.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:41.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.697+0000 7f515c132980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 2024-03-04T18:06:41.702 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.705+0000 7f515c132980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:41.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.705+0000 7f515c132980 10 moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:41.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.705+0000 7f515c132980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:41.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.705+0000 7f515c132980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18, ofs0, ns] 2024-03-04T18:06:41.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.705+0000 7f503d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595112 cookie 94461378792464 notifier 7056 bl.length()=783 2024-03-04T18:06:41.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.705+0000 7f503d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:41.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.705+0000 7f503d7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:41.704 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.705+0000 7f503d7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:41.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.709+0000 7f515c132980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_sources= new_sources= 2024-03-04T18:06:41.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.709+0000 7f515c132980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential sources added=[] removed=[] 2024-03-04T18:06:41.706 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.709+0000 7f515c132980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_dests= new_dests= 2024-03-04T18:06:41.707 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.709+0000 7f515c132980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential dests added=[] removed=[] 2024-03-04T18:06:41.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.0 2024-03-04T18:06:41.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.0 2024-03-04T18:06:41.713 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.1 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.1 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.2 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.2 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.3 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.3 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.4 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.4 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.5 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.5 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.6 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.6 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.7 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.7 2024-03-04T18:06:41.714 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.8 2024-03-04T18:06:41.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.8 2024-03-04T18:06:41.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.9 2024-03-04T18:06:41.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.9 2024-03-04T18:06:41.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.10 2024-03-04T18:06:41.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.10 2024-03-04T18:06:41.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.11 2024-03-04T18:06:41.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.11 2024-03-04T18:06:41.715 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.0 2024-03-04T18:06:41.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1 2024-03-04T18:06:41.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2 2024-03-04T18:06:41.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3 2024-03-04T18:06:41.716 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.717+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.4 2024-03-04T18:06:41.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.721+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.5 2024-03-04T18:06:41.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.721+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.6 2024-03-04T18:06:41.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.721+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.7 2024-03-04T18:06:41.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.721+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.8 2024-03-04T18:06:41.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.721+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.9 2024-03-04T18:06:41.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.725+0000 7f515c132980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.10 2024-03-04T18:06:41.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.725+0000 7f515c132980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:41.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.725+0000 7f515c132980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 2024-03-04T18:06:41.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.725+0000 7f515c132980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:41.723 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.725+0000 7f515c132980 10 moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:41.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.725+0000 7f515c132980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:41.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.725+0000 7f515c132980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18, ofs0, ns] 2024-03-04T18:06:41.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.729+0000 7f503d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595113 cookie 94461378792464 notifier 7056 bl.length()=751 2024-03-04T18:06:41.724 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.729+0000 7f503d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:41.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.729+0000 7f503d7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:41.725 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.729+0000 7f503d7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:41.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.729+0000 7f515c132980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_sources= new_sources= 2024-03-04T18:06:41.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.729+0000 7f515c132980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential sources added=[] removed=[] 2024-03-04T18:06:41.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.729+0000 7f515c132980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_dests= new_dests= 2024-03-04T18:06:41.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.729+0000 7f515c132980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential dests added=[] removed=[] 2024-03-04T18:06:41.732 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.733+0000 7f515c132980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-03-04T18:06:41.734 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.737+0000 7f500b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:41.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.741+0000 7f515c132980 20 remove_watcher() i=0 2024-03-04T18:06:41.738 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.741+0000 7f515c132980 2 removed watcher, disabling cache 2024-03-04T18:06:41.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.741+0000 7f515c132980 20 remove_watcher() i=1 2024-03-04T18:06:41.742 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.745+0000 7f515c132980 20 remove_watcher() i=2 2024-03-04T18:06:41.744 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.749+0000 7f515c132980 20 remove_watcher() i=3 2024-03-04T18:06:41.747 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.749+0000 7f515c132980 20 remove_watcher() i=4 2024-03-04T18:06:41.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.753+0000 7f515c132980 20 remove_watcher() i=5 2024-03-04T18:06:41.751 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.753+0000 7f515c132980 20 remove_watcher() i=6 2024-03-04T18:06:41.754 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:41.757+0000 7f515c132980 20 remove_watcher() i=7 2024-03-04T18:06:41.769 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket myver 2024-03-04T18:06:41.798 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:41.798 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:42.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.053+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.057+0000 7f2dbebb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.057+0000 7f2dbebb2980 20 realm 2024-03-04T18:06:42.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.057+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.057+0000 7f2dbebb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.057+0000 7f2dbebb2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:42.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.057+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.057+0000 7f2dbebb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.056 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.057+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.061+0000 7f2dbebb2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:42.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.061+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.061+0000 7f2dbebb2980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:42.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.061+0000 7f2dbebb2980 20 searching for the correct realm 2024-03-04T18:06:42.064 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.065+0000 7f2dbbec9640 0 -- 172.21.15.47:0/1603137610 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f2db0068a90 msgr2=0x7f2db0064050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:42.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.277+0000 7f2dbcecb640 0 -- 172.21.15.47:0/1603137610 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x556407dc4180 msgr2=0x556407dc6570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:42.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:42.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:42.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:42.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:42.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:42.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:42.289 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.290 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:42.291 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:42.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 zone default found 2024-03-04T18:06:42.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 4 Realm: () 2024-03-04T18:06:42.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:42.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:42.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:42.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 zonegroup default 2024-03-04T18:06:42.292 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.293+0000 7f2dbebb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.297+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.297+0000 7f2dbebb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.293 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.297+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.294 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.297+0000 7f2dbebb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.297+0000 7f2dbebb2980 20 started sync module instance, tier type = 2024-03-04T18:06:42.295 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.297+0000 7f2dbebb2980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:42.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.321+0000 7f2dbebb2980 20 add_watcher() i=0 2024-03-04T18:06:42.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.321+0000 7f2dbebb2980 20 add_watcher() i=1 2024-03-04T18:06:42.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.321+0000 7f2dbebb2980 20 add_watcher() i=2 2024-03-04T18:06:42.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.321+0000 7f2dbebb2980 20 add_watcher() i=3 2024-03-04T18:06:42.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.321+0000 7f2dbebb2980 20 add_watcher() i=4 2024-03-04T18:06:42.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.321+0000 7f2dbebb2980 20 add_watcher() i=5 2024-03-04T18:06:42.320 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.321+0000 7f2dbebb2980 20 add_watcher() i=6 2024-03-04T18:06:42.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.325+0000 7f2dbebb2980 20 add_watcher() i=7 2024-03-04T18:06:42.322 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.325+0000 7f2dbebb2980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:42.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.329+0000 7f2c72fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:42.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.329+0000 7f2dbebb2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:42.327 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.329+0000 7f2dbebb2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:42.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.329+0000 7f2dbebb2980 5 note: GC not initialized 2024-03-04T18:06:42.328 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.329+0000 7f2c71fdb640 20 reqs_thread_entry: start 2024-03-04T18:06:42.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.329+0000 7f2dbebb2980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:42.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.329+0000 7f2dbebb2980 20 Filter name: none 2024-03-04T18:06:42.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.329+0000 7f2be37fe640 20 reqs_thread_entry: start 2024-03-04T18:06:42.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.329+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.333+0000 7f2dbebb2980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:42.329 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.333+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.330 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.333+0000 7f2dbebb2980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:42.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.333+0000 7f2dbebb2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:42.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.333+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.333+0000 7f2dbebb2980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:42.331 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.333+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.333+0000 7f2dbebb2980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:42.332 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.333+0000 7f2dbebb2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:42.334 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.337+0000 7f2dbebb2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.335 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.337+0000 7f2dbebb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.336 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.337+0000 7f2c72fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:42.340 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.345+0000 7f2dbebb2980 20 remove_watcher() i=0 2024-03-04T18:06:42.341 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.345+0000 7f2dbebb2980 2 removed watcher, disabling cache 2024-03-04T18:06:42.343 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.345+0000 7f2dbebb2980 20 remove_watcher() i=1 2024-03-04T18:06:42.345 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.349+0000 7f2dbebb2980 20 remove_watcher() i=2 2024-03-04T18:06:42.347 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.349+0000 7f2dbebb2980 20 remove_watcher() i=3 2024-03-04T18:06:42.349 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.353+0000 7f2dbebb2980 20 remove_watcher() i=4 2024-03-04T18:06:42.352 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.353+0000 7f2dbebb2980 20 remove_watcher() i=5 2024-03-04T18:06:42.354 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.357+0000 7f2dbebb2980 20 remove_watcher() i=6 2024-03-04T18:06:42.357 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.361+0000 7f2dbebb2980 20 remove_watcher() i=7 2024-03-04T18:06:42.372 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:42.372 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:42.373 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: myver id: 797b67a4-b876-4d04-b122-10afd858d465.4453.18 num_objects: 0 size_kb: 0 num_shards: 12 2024-03-04T18:06:42.373 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:42.375 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:42.375 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:42.401 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:42.402 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:42.457 INFO:tasks.workunit.client.0.smithi047.stderr: test: reshard removes olh entries with empty name 2024-03-04T18:06:42.462 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:42.463 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:42.598 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:42.598 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:42.848 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:06:42.882 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:42.882 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:42.927 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.929+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 20 realm 2024-03-04T18:06:42.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.930 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:42.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:42.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:42.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.933+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:42.932 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.937+0000 7fd3d2d6f980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:42.933 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.937+0000 7fd3d2d6f980 20 searching for the correct realm 2024-03-04T18:06:42.937 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:42.941+0000 7fd3d0887640 0 -- 172.21.15.47:0/755896398 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fd3c0045060 msgr2=0x7fd3c0047790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:43.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.153+0000 7fd3d0887640 0 -- 172.21.15.47:0/755896398 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fd3c8008040 msgr2=0x7fd3c805ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:43.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:43.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:43.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:43.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:43.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:43.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:43.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:43.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:43.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 zone default found 2024-03-04T18:06:43.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 4 Realm: () 2024-03-04T18:06:43.371 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:43.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:43.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:43.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 zonegroup default 2024-03-04T18:06:43.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.372 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.373+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.377+0000 7fd3d2d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.373 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.377+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.377+0000 7fd3d2d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.377+0000 7fd3d2d6f980 20 started sync module instance, tier type = 2024-03-04T18:06:43.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.377+0000 7fd3d2d6f980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:43.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.401+0000 7fd3d2d6f980 20 add_watcher() i=0 2024-03-04T18:06:43.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.401+0000 7fd3d2d6f980 20 add_watcher() i=1 2024-03-04T18:06:43.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.401+0000 7fd3d2d6f980 20 add_watcher() i=2 2024-03-04T18:06:43.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.401+0000 7fd3d2d6f980 20 add_watcher() i=3 2024-03-04T18:06:43.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.401+0000 7fd3d2d6f980 20 add_watcher() i=4 2024-03-04T18:06:43.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.401+0000 7fd3d2d6f980 20 add_watcher() i=5 2024-03-04T18:06:43.398 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.401+0000 7fd3d2d6f980 20 add_watcher() i=6 2024-03-04T18:06:43.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.401+0000 7fd3d2d6f980 20 add_watcher() i=7 2024-03-04T18:06:43.400 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.401+0000 7fd3d2d6f980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:43.404 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.405+0000 7fd286fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:43.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:43.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:43.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 5 note: GC not initialized 2024-03-04T18:06:43.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd285fe3640 20 reqs_thread_entry: start 2024-03-04T18:06:43.405 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:43.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 Filter name: none 2024-03-04T18:06:43.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd1f77fe640 20 reqs_thread_entry: start 2024-03-04T18:06:43.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:06:43.407 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:06:43.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:43.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.409+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.413+0000 7fd3d2d6f980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:06:43.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.413+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.413+0000 7fd3d2d6f980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:06:43.409 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.413+0000 7fd3d2d6f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:43.411 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.413+0000 7fd3d2d6f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.412 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.413+0000 7fd3d2d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.417+0000 7fd286fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:43.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.421+0000 7fd3d2d6f980 20 remove_watcher() i=0 2024-03-04T18:06:43.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.421+0000 7fd3d2d6f980 2 removed watcher, disabling cache 2024-03-04T18:06:43.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.421+0000 7fd3d2d6f980 20 remove_watcher() i=1 2024-03-04T18:06:43.423 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.425+0000 7fd3d2d6f980 20 remove_watcher() i=2 2024-03-04T18:06:43.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.429+0000 7fd3d2d6f980 20 remove_watcher() i=3 2024-03-04T18:06:43.427 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.429+0000 7fd3d2d6f980 20 remove_watcher() i=4 2024-03-04T18:06:43.429 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.433+0000 7fd3d2d6f980 20 remove_watcher() i=5 2024-03-04T18:06:43.432 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.433+0000 7fd3d2d6f980 20 remove_watcher() i=6 2024-03-04T18:06:43.435 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.437+0000 7fd3d2d6f980 20 remove_watcher() i=7 2024-03-04T18:06:43.450 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:43.450 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:43.450 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: a-bucket id: 797b67a4-b876-4d04-b122-10afd858d465.4453.17 num_objects: 0 size_kb: 0 num_shards: 13 2024-03-04T18:06:43.450 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:43.450 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket layout --bucket a-bucket 2024-03-04T18:06:43.482 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:43.483 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:43.526 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.529+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 realm 2024-03-04T18:06:43.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:43.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:43.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:43.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.533+0000 7f924fb3c980 20 searching for the correct realm 2024-03-04T18:06:43.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.541+0000 7f924de55640 0 -- 172.21.15.47:0/915945631 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f9244038070 msgr2=0x7f9244038960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:43.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.549+0000 7f924d654640 0 -- 172.21.15.47:0/915945631 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f92480bb040 msgr2=0x7f92480f1e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:43.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:43.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:43.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:43.553 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:43.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:43.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:43.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:43.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:43.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 zone default found 2024-03-04T18:06:43.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 4 Realm: () 2024-03-04T18:06:43.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:43.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:43.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:43.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 zonegroup default 2024-03-04T18:06:43.555 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.557+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.561+0000 7f924fb3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.561+0000 7f924fb3c980 20 started sync module instance, tier type = 2024-03-04T18:06:43.557 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.561+0000 7f924fb3c980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:43.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.577+0000 7f924fb3c980 20 add_watcher() i=0 2024-03-04T18:06:43.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.577+0000 7f924fb3c980 20 add_watcher() i=1 2024-03-04T18:06:43.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.577+0000 7f924fb3c980 20 add_watcher() i=2 2024-03-04T18:06:43.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.577+0000 7f924fb3c980 20 add_watcher() i=3 2024-03-04T18:06:43.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.577+0000 7f924fb3c980 20 add_watcher() i=4 2024-03-04T18:06:43.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.577+0000 7f924fb3c980 20 add_watcher() i=5 2024-03-04T18:06:43.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.577+0000 7f924fb3c980 20 add_watcher() i=6 2024-03-04T18:06:43.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.581+0000 7f924fb3c980 20 add_watcher() i=7 2024-03-04T18:06:43.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.581+0000 7f924fb3c980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:43.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.581+0000 7f9103fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:43.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:43.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:43.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 5 note: GC not initialized 2024-03-04T18:06:43.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f9102fe5640 20 reqs_thread_entry: start 2024-03-04T18:06:43.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:43.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 20 Filter name: none 2024-03-04T18:06:43.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f9100fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:43.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:06:43.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:06:43.582 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f924fb3c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:43.583 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.585+0000 7f9103fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:43.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.589+0000 7f924fb3c980 20 remove_watcher() i=0 2024-03-04T18:06:43.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.589+0000 7f924fb3c980 2 removed watcher, disabling cache 2024-03-04T18:06:43.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.593+0000 7f924fb3c980 20 remove_watcher() i=1 2024-03-04T18:06:43.591 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.593+0000 7f924fb3c980 20 remove_watcher() i=2 2024-03-04T18:06:43.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.597+0000 7f924fb3c980 20 remove_watcher() i=3 2024-03-04T18:06:43.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.597+0000 7f924fb3c980 20 remove_watcher() i=4 2024-03-04T18:06:43.598 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.601+0000 7f924fb3c980 20 remove_watcher() i=5 2024-03-04T18:06:43.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.605+0000 7f924fb3c980 20 remove_watcher() i=6 2024-03-04T18:06:43.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.605+0000 7f924fb3c980 20 remove_watcher() i=7 2024-03-04T18:06:43.620 INFO:tasks.workunit.client.0.smithi047.stderr:rados -p default.rgw.buckets.index setomapval .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.0 --omap-key-file /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_key < /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_val 2024-03-04T18:06:43.650 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:43.650 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:43.670 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket a-bucket 2024-03-04T18:06:43.696 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:43.696 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:43.737 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.741+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.741+0000 7efe03dbe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.741+0000 7efe03dbe980 20 realm 2024-03-04T18:06:43.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.741+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.745+0000 7efe03dbe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.745+0000 7efe03dbe980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:43.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.745+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.745+0000 7efe03dbe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.745+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.742 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.745+0000 7efe03dbe980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:43.742 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.745+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.743 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.745+0000 7efe03dbe980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:43.743 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.745+0000 7efe03dbe980 20 searching for the correct realm 2024-03-04T18:06:43.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.753+0000 7efe010d5640 0 -- 172.21.15.47:0/3969153797 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7efdf8008420 msgr2=0x7efdf80088b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:43.755 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.757+0000 7efe018d6640 0 -- 172.21.15.47:0/3969153797 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7efdfc07e2e0 msgr2=0x7efdfc056090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:43.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:43.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:43.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:43.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:43.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:43.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:43.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:43.764 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 zone default found 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 4 Realm: () 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 zonegroup default 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.765+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.769+0000 7efe03dbe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.765 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.769+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.769+0000 7efe03dbe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.769+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.769+0000 7efe03dbe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.769+0000 7efe03dbe980 20 started sync module instance, tier type = 2024-03-04T18:06:43.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.769+0000 7efe03dbe980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:43.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.785+0000 7efe03dbe980 20 add_watcher() i=0 2024-03-04T18:06:43.783 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.785+0000 7efe03dbe980 20 add_watcher() i=1 2024-03-04T18:06:43.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.785+0000 7efe03dbe980 20 add_watcher() i=2 2024-03-04T18:06:43.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.785+0000 7efe03dbe980 20 add_watcher() i=3 2024-03-04T18:06:43.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.785+0000 7efe03dbe980 20 add_watcher() i=4 2024-03-04T18:06:43.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.785+0000 7efe03dbe980 20 add_watcher() i=5 2024-03-04T18:06:43.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.785+0000 7efe03dbe980 20 add_watcher() i=6 2024-03-04T18:06:43.786 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.789+0000 7efe03dbe980 20 add_watcher() i=7 2024-03-04T18:06:43.786 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.789+0000 7efe03dbe980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:43.789 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efcb3fff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:43.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:43.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:43.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 5 note: GC not initialized 2024-03-04T18:06:43.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efcb2ffd640 20 reqs_thread_entry: start 2024-03-04T18:06:43.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:43.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 Filter name: none 2024-03-04T18:06:43.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efcb0ff9640 20 reqs_thread_entry: start 2024-03-04T18:06:43.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:06:43.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:06:43.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:43.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-03-04T18:06:43.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 INFO: main: starting shard=0 2024-03-04T18:06:43.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.0 2024-03-04T18:06:43.792 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.793+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.0 2024-03-04T18:06:43.793 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T18:06:43.793 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: starting shard=1 2024-03-04T18:06:43.793 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.1 2024-03-04T18:06:43.793 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.1 2024-03-04T18:06:43.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: starting shard=2 2024-03-04T18:06:43.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.2 2024-03-04T18:06:43.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.2 2024-03-04T18:06:43.794 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: starting shard=3 2024-03-04T18:06:43.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.3 2024-03-04T18:06:43.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.3 2024-03-04T18:06:43.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: starting shard=4 2024-03-04T18:06:43.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.4 2024-03-04T18:06:43.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.4 2024-03-04T18:06:43.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 INFO: main: starting shard=5 2024-03-04T18:06:43.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.5 2024-03-04T18:06:43.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.797+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.5 2024-03-04T18:06:43.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.797 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 INFO: main: starting shard=6 2024-03-04T18:06:43.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.6 2024-03-04T18:06:43.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.6 2024-03-04T18:06:43.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 INFO: main: starting shard=7 2024-03-04T18:06:43.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.7 2024-03-04T18:06:43.798 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.7 2024-03-04T18:06:43.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 INFO: main: starting shard=8 2024-03-04T18:06:43.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.8 2024-03-04T18:06:43.799 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.8 2024-03-04T18:06:43.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.801+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 INFO: main: starting shard=9 2024-03-04T18:06:43.800 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.9 2024-03-04T18:06:43.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.9 2024-03-04T18:06:43.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 INFO: main: starting shard=10 2024-03-04T18:06:43.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.10 2024-03-04T18:06:43.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.10 2024-03-04T18:06:43.801 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 INFO: main: starting shard=11 2024-03-04T18:06:43.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.11 2024-03-04T18:06:43.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.11 2024-03-04T18:06:43.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 INFO: main: starting shard=12 2024-03-04T18:06:43.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.12 2024-03-04T18:06:43.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.12 2024-03-04T18:06:43.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:43.804 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.805+0000 7efe03dbe980 20 INFO: main: done 2024-03-04T18:06:43.805 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.809+0000 7efcb3fff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:43.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.813+0000 7efe03dbe980 20 remove_watcher() i=0 2024-03-04T18:06:43.809 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.813+0000 7efe03dbe980 2 removed watcher, disabling cache 2024-03-04T18:06:43.811 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.813+0000 7efe03dbe980 20 remove_watcher() i=1 2024-03-04T18:06:43.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.817+0000 7efe03dbe980 20 remove_watcher() i=2 2024-03-04T18:06:43.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.817+0000 7efe03dbe980 20 remove_watcher() i=3 2024-03-04T18:06:43.817 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.821+0000 7efe03dbe980 20 remove_watcher() i=4 2024-03-04T18:06:43.820 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.821+0000 7efe03dbe980 20 remove_watcher() i=5 2024-03-04T18:06:43.823 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.825+0000 7efe03dbe980 20 remove_watcher() i=6 2024-03-04T18:06:43.825 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.829+0000 7efe03dbe980 20 remove_watcher() i=7 2024-03-04T18:06:43.840 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 1 --yes-i-really-mean-it 2024-03-04T18:06:43.869 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:43.870 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:43.913 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.917+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.917+0000 7f825c928980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.915 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.917+0000 7f825c928980 20 realm 2024-03-04T18:06:43.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.917+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.917+0000 7f825c928980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.917+0000 7f825c928980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:43.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.917+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.916 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.921+0000 7f825c928980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.921+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.921+0000 7f825c928980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:43.917 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.921+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.921+0000 7f825c928980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:43.918 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.921+0000 7f825c928980 20 searching for the correct realm 2024-03-04T18:06:43.925 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.929+0000 7f825a440640 0 -- 172.21.15.47:0/279143254 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f824c08fb60 msgr2=0x7f824c067040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:43.931 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.933+0000 7f825a440640 0 -- 172.21.15.47:0/279143254 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f82540ab070 msgr2=0x7f82540d33f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:43.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:43.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:43.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:43.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:43.938 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:43.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:43.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:43.939 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:43.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 zone default found 2024-03-04T18:06:43.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 4 Realm: () 2024-03-04T18:06:43.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:43.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:43.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:43.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 zonegroup default 2024-03-04T18:06:43.940 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.941+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.945+0000 7f825c928980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.945+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.945+0000 7f825c928980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:43.941 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.945+0000 7f825c928980 20 started sync module instance, tier type = 2024-03-04T18:06:43.942 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.945+0000 7f825c928980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:43.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.961+0000 7f825c928980 20 add_watcher() i=0 2024-03-04T18:06:43.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.961+0000 7f825c928980 20 add_watcher() i=1 2024-03-04T18:06:43.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.961+0000 7f825c928980 20 add_watcher() i=2 2024-03-04T18:06:43.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.961+0000 7f825c928980 20 add_watcher() i=3 2024-03-04T18:06:43.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.961+0000 7f825c928980 20 add_watcher() i=4 2024-03-04T18:06:43.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.961+0000 7f825c928980 20 add_watcher() i=5 2024-03-04T18:06:43.960 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.961+0000 7f825c928980 20 add_watcher() i=6 2024-03-04T18:06:43.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.965+0000 7f825c928980 20 add_watcher() i=7 2024-03-04T18:06:43.962 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.965+0000 7f825c928980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:43.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f8110fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:43.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:43.966 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:43.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 5 note: GC not initialized 2024-03-04T18:06:43.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f808b7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:43.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:43.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 20 Filter name: none 2024-03-04T18:06:43.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f80897fa640 20 reqs_thread_entry: start 2024-03-04T18:06:43.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:06:43.967 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:06:43.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-03-04T18:06:43.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:06:43.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 : miss 2024-03-04T18:06:43.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 20 rados->read ofs=0 len=0 2024-03-04T18:06:43.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:06:43.968 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:06:43.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:06:43.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:43.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:43.969 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.969+0000 7f825c928980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 2024-03-04T18:06:43.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f825c928980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:06:43.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f825c928980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:06:43.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f825c928980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:43.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f825c928980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17, ofs0, ns] 2024-03-04T18:06:43.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f813d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595120 cookie 94840463270240 notifier 6713 bl.length()=789 2024-03-04T18:06:43.974 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f813d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:06:43.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f813d7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:06:43.975 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f813d7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:43.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f825c928980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_sources= new_sources= 2024-03-04T18:06:43.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f825c928980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential sources added=[] removed=[] 2024-03-04T18:06:43.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f825c928980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_dests= new_dests= 2024-03-04T18:06:43.976 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.977+0000 7f825c928980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential dests added=[] removed=[] 2024-03-04T18:06:43.982 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.985+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.3.0 2024-03-04T18:06:43.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.985+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.3.0 2024-03-04T18:06:43.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.985+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.0 2024-03-04T18:06:43.983 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.985+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.0 2024-03-04T18:06:43.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.985+0000 7f825c928980 10 Dropping entry with empty name, idx=�1001_04/57/0457f727ec113e418d5b16d206b200ed068c0533554883ce811df7c932a3df68/2018_12_11/2889999/3386469/metadata.gz 2024-03-04T18:06:43.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.989+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.1 2024-03-04T18:06:43.984 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.989+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.1 2024-03-04T18:06:43.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.989+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.2 2024-03-04T18:06:43.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.989+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.2 2024-03-04T18:06:43.986 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.989+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.3 2024-03-04T18:06:43.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.989+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.3 2024-03-04T18:06:43.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.989+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.4 2024-03-04T18:06:43.987 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.989+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.4 2024-03-04T18:06:43.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.993+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.5 2024-03-04T18:06:43.989 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.993+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.5 2024-03-04T18:06:43.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.993+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.6 2024-03-04T18:06:43.990 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.993+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.6 2024-03-04T18:06:43.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.993+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.7 2024-03-04T18:06:43.991 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.993+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.7 2024-03-04T18:06:43.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.993+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.8 2024-03-04T18:06:43.992 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.993+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.8 2024-03-04T18:06:43.993 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.997+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.9 2024-03-04T18:06:43.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.997+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.9 2024-03-04T18:06:43.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.997+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.10 2024-03-04T18:06:43.994 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.997+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.10 2024-03-04T18:06:43.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.997+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.11 2024-03-04T18:06:43.995 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.997+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.11 2024-03-04T18:06:43.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.997+0000 7f825c928980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.12 2024-03-04T18:06:43.996 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:43.997+0000 7f825c928980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.2.12 2024-03-04T18:06:43.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f825c928980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:43.997 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f825c928980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 2024-03-04T18:06:43.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f825c928980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:06:43.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f825c928980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:06:43.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f825c928980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:43.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f825c928980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17, ofs0, ns] 2024-03-04T18:06:43.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f813dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595121 cookie 94840463270240 notifier 6713 bl.length()=757 2024-03-04T18:06:43.999 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f813dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 info.flags=0x17 2024-03-04T18:06:44.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f813dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:797b67a4-b876-4d04-b122-10afd858d465.4453.17 to cache LRU end 2024-03-04T18:06:44.000 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.001+0000 7f813dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:44.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.005+0000 7f825c928980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_sources= new_sources= 2024-03-04T18:06:44.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.005+0000 7f825c928980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential sources added=[] removed=[] 2024-03-04T18:06:44.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.005+0000 7f825c928980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): orig_dests= new_dests= 2024-03-04T18:06:44.001 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.005+0000 7f825c928980 20 handle_bi_update(): bucket=:a-bucket[797b67a4-b876-4d04-b122-10afd858d465.4453.17]): potential dests added=[] removed=[] 2024-03-04T18:06:44.007 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.009+0000 7f825c928980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-03-04T18:06:44.010 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.013+0000 7f8110fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:44.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.017+0000 7f825c928980 20 remove_watcher() i=0 2024-03-04T18:06:44.014 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.017+0000 7f825c928980 2 removed watcher, disabling cache 2024-03-04T18:06:44.017 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.021+0000 7f825c928980 20 remove_watcher() i=1 2024-03-04T18:06:44.019 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.021+0000 7f825c928980 20 remove_watcher() i=2 2024-03-04T18:06:44.021 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.025+0000 7f825c928980 20 remove_watcher() i=3 2024-03-04T18:06:44.023 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.025+0000 7f825c928980 20 remove_watcher() i=4 2024-03-04T18:06:44.025 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.029+0000 7f825c928980 20 remove_watcher() i=5 2024-03-04T18:06:44.028 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.029+0000 7f825c928980 20 remove_watcher() i=6 2024-03-04T18:06:44.031 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.033+0000 7f825c928980 20 remove_watcher() i=7 2024-03-04T18:06:44.046 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket a-bucket 2024-03-04T18:06:44.075 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:44.076 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:44.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.121+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.125+0000 7fd611579980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.125+0000 7fd611579980 20 realm 2024-03-04T18:06:44.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.125+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.125+0000 7fd611579980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.125+0000 7fd611579980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:44.123 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.125+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.125+0000 7fd611579980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.125+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.129+0000 7fd611579980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:44.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.129+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.129+0000 7fd611579980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:44.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.129+0000 7fd611579980 20 searching for the correct realm 2024-03-04T18:06:44.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.133+0000 7fd60f091640 0 -- 172.21.15.47:0/1494544633 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fd600040070 msgr2=0x7fd600053040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:44.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.141+0000 7fd60f091640 0 -- 172.21.15.47:0/1494544633 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fd6080ba040 msgr2=0x7fd6080a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:44.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:44.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:44.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:44.358 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:44.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:44.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:44.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:44.359 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.361+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.360 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:44.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 zone default found 2024-03-04T18:06:44.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 4 Realm: () 2024-03-04T18:06:44.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:44.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:44.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:44.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 zonegroup default 2024-03-04T18:06:44.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.361 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.362 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 started sync module instance, tier type = 2024-03-04T18:06:44.363 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.365+0000 7fd611579980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:44.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.389+0000 7fd611579980 20 add_watcher() i=0 2024-03-04T18:06:44.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.389+0000 7fd611579980 20 add_watcher() i=1 2024-03-04T18:06:44.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.389+0000 7fd611579980 20 add_watcher() i=2 2024-03-04T18:06:44.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.389+0000 7fd611579980 20 add_watcher() i=3 2024-03-04T18:06:44.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.389+0000 7fd611579980 20 add_watcher() i=4 2024-03-04T18:06:44.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.389+0000 7fd611579980 20 add_watcher() i=5 2024-03-04T18:06:44.386 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.389+0000 7fd611579980 20 add_watcher() i=6 2024-03-04T18:06:44.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.389+0000 7fd611579980 20 add_watcher() i=7 2024-03-04T18:06:44.388 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.389+0000 7fd611579980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:44.392 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd4c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:44.393 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:44.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:44.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 5 note: GC not initialized 2024-03-04T18:06:44.394 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd4c4fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:44.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:44.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 Filter name: none 2024-03-04T18:06:44.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd43e7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:44.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 rados_obj.operate() r=0 bl.length=176 2024-03-04T18:06:44.395 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 rados_obj.operate() r=0 bl.length=360 2024-03-04T18:06:44.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:44.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 INFO: main: max_entries=1000, index=gen=3, max_shards=1 2024-03-04T18:06:44.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 INFO: main: starting shard=0 2024-03-04T18:06:44.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.3.0 2024-03-04T18:06:44.396 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.397+0000 7fd611579980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.17.3.0 2024-03-04T18:06:44.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.401+0000 7fd611579980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:44.397 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.401+0000 7fd611579980 20 INFO: main: done 2024-03-04T18:06:44.399 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.401+0000 7fd4c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:44.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.405+0000 7fd611579980 20 remove_watcher() i=0 2024-03-04T18:06:44.403 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.405+0000 7fd611579980 2 removed watcher, disabling cache 2024-03-04T18:06:44.406 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.409+0000 7fd611579980 20 remove_watcher() i=1 2024-03-04T18:06:44.408 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.409+0000 7fd611579980 20 remove_watcher() i=2 2024-03-04T18:06:44.410 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.413+0000 7fd611579980 20 remove_watcher() i=3 2024-03-04T18:06:44.413 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.417+0000 7fd611579980 20 remove_watcher() i=4 2024-03-04T18:06:44.415 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.417+0000 7fd611579980 20 remove_watcher() i=5 2024-03-04T18:06:44.418 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.421+0000 7fd611579980 20 remove_watcher() i=6 2024-03-04T18:06:44.421 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.425+0000 7fd611579980 20 remove_watcher() i=7 2024-03-04T18:06:44.440 INFO:tasks.workunit.client.0.smithi047.stderr: test: PUT succeeds during reshard 2024-03-04T18:06:44.440 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket myver 2024-03-04T18:06:44.474 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:44.474 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:44.518 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.521+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 realm 2024-03-04T18:06:44.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:44.521 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:44.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:44.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.525+0000 7efd59700980 20 searching for the correct realm 2024-03-04T18:06:44.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.533+0000 7efd56a17640 0 -- 172.21.15.47:0/1417456086 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7efd4c02e120 msgr2=0x7efd4c05af70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:44.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.541+0000 7efd57a19640 0 -- 172.21.15.47:0/1417456086 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x5624e4349280 msgr2=0x5624e434b6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:44.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:44.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:44.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:44.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:44.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:44.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:44.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:44.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:44.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 zone default found 2024-03-04T18:06:44.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 4 Realm: () 2024-03-04T18:06:44.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:44.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:44.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:44.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 zonegroup default 2024-03-04T18:06:44.548 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.549+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.553+0000 7efd59700980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.553+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.553+0000 7efd59700980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.553+0000 7efd59700980 20 started sync module instance, tier type = 2024-03-04T18:06:44.550 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.553+0000 7efd59700980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:44.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.569+0000 7efd59700980 20 add_watcher() i=0 2024-03-04T18:06:44.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.569+0000 7efd59700980 20 add_watcher() i=1 2024-03-04T18:06:44.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.569+0000 7efd59700980 20 add_watcher() i=2 2024-03-04T18:06:44.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.569+0000 7efd59700980 20 add_watcher() i=3 2024-03-04T18:06:44.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.569+0000 7efd59700980 20 add_watcher() i=4 2024-03-04T18:06:44.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.569+0000 7efd59700980 20 add_watcher() i=5 2024-03-04T18:06:44.568 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.569+0000 7efd59700980 20 add_watcher() i=6 2024-03-04T18:06:44.570 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.573+0000 7efd59700980 20 add_watcher() i=7 2024-03-04T18:06:44.571 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.573+0000 7efd59700980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:44.574 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efc0dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:44.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efd59700980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:44.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efd59700980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:44.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efd59700980 5 note: GC not initialized 2024-03-04T18:06:44.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efc0cfe1640 20 reqs_thread_entry: start 2024-03-04T18:06:44.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efd59700980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:44.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efd59700980 20 Filter name: none 2024-03-04T18:06:44.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efb867fc640 20 reqs_thread_entry: start 2024-03-04T18:06:44.575 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efd59700980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:44.576 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.577+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.581+0000 7efd59700980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:44.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.581+0000 7efd59700980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:44.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.581+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.581+0000 7efd59700980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:44.577 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.581+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.581+0000 7efd59700980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:44.578 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.581+0000 7efd59700980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:44.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.581+0000 7efd59700980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.580 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.585+0000 7efd59700980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.581 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.585+0000 7efc0dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:44.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.589+0000 7efd59700980 20 remove_watcher() i=0 2024-03-04T18:06:44.585 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.589+0000 7efd59700980 2 removed watcher, disabling cache 2024-03-04T18:06:44.587 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.589+0000 7efd59700980 20 remove_watcher() i=1 2024-03-04T18:06:44.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.593+0000 7efd59700980 20 remove_watcher() i=2 2024-03-04T18:06:44.592 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.593+0000 7efd59700980 20 remove_watcher() i=3 2024-03-04T18:06:44.594 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.597+0000 7efd59700980 20 remove_watcher() i=4 2024-03-04T18:06:44.596 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.597+0000 7efd59700980 20 remove_watcher() i=5 2024-03-04T18:06:44.599 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.601+0000 7efd59700980 20 remove_watcher() i=6 2024-03-04T18:06:44.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.605+0000 7efd59700980 20 remove_watcher() i=7 2024-03-04T18:06:44.616 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:44.617 INFO:tasks.workunit.client.0.smithi047.stderr:BUCKET_STATS: 2024-03-04T18:06:44.617 INFO:tasks.workunit.client.0.smithi047.stderr:bucket: myver id: 797b67a4-b876-4d04-b122-10afd858d465.4453.18 num_objects: 0 size_kb: 0 num_shards: 12 2024-03-04T18:06:44.617 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:44.617 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin --inject-delay-at=do_reshard --inject-delay-ms=5000 bucket reshard --bucket myver --num-shards 13 2024-03-04T18:06:44.648 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:44.648 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:44.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.693+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 20 realm 2024-03-04T18:06:44.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.694 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:44.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:44.696 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.697+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.697 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.701+0000 7f28f5732980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:44.697 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.701+0000 7f28f5732980 20 searching for the correct realm 2024-03-04T18:06:44.703 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.705+0000 7f28f2a49640 0 -- 172.21.15.47:0/1959921942 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f28e80c1410 msgr2=0x7f28e80c1890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:44.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.713+0000 7f28f324a640 0 -- 172.21.15.47:0/1959921942 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f28ec020070 msgr2=0x7f28ec06dab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:44.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:44.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:44.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:44.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:44.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:44.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:44.718 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:44.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:44.719 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 zone default found 2024-03-04T18:06:44.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 4 Realm: () 2024-03-04T18:06:44.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:44.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:44.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:44.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 zonegroup default 2024-03-04T18:06:44.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.721+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.725+0000 7f28f5732980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:44.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.725+0000 7f28f5732980 20 started sync module instance, tier type = 2024-03-04T18:06:44.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.725+0000 7f28f5732980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:44.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.741+0000 7f28f5732980 20 add_watcher() i=0 2024-03-04T18:06:44.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.741+0000 7f28f5732980 20 add_watcher() i=1 2024-03-04T18:06:44.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.741+0000 7f28f5732980 20 add_watcher() i=2 2024-03-04T18:06:44.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.741+0000 7f28f5732980 20 add_watcher() i=3 2024-03-04T18:06:44.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.741+0000 7f28f5732980 20 add_watcher() i=4 2024-03-04T18:06:44.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.741+0000 7f28f5732980 20 add_watcher() i=5 2024-03-04T18:06:44.739 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.741+0000 7f28f5732980 20 add_watcher() i=6 2024-03-04T18:06:44.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.745+0000 7f28f5732980 20 add_watcher() i=7 2024-03-04T18:06:44.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.745+0000 7f28f5732980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:44.745 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f27a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:44.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:44.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:44.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 5 note: GC not initialized 2024-03-04T18:06:44.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f27a8fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:44.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:44.746 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 20 Filter name: none 2024-03-04T18:06:44.747 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f27227fc640 20 reqs_thread_entry: start 2024-03-04T18:06:44.747 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-03-04T18:06:44.747 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:44.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-03-04T18:06:44.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-03-04T18:06:44.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 : miss 2024-03-04T18:06:44.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 20 rados->read ofs=0 len=0 2024-03-04T18:06:44.748 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:44.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:44.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 10 adding default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:44.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:44.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:44.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.749+0000 7f28f5732980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 2024-03-04T18:06:44.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.761+0000 7f28f5732980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:44.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.761+0000 7f28f5732980 10 moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:44.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.761+0000 7f28f5732980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:44.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.761+0000 7f28f5732980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18, ofs0, ns] 2024-03-04T18:06:44.760 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.765+0000 7f27d6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595114 cookie 94430016869888 notifier 6764 bl.length()=783 2024-03-04T18:06:44.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.765+0000 7f27d6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:44.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.765+0000 7f27d6ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:44.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.765+0000 7f27d6ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:44.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.765+0000 7f28f5732980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_sources= new_sources= 2024-03-04T18:06:44.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.765+0000 7f28f5732980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential sources added=[] removed=[] 2024-03-04T18:06:44.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.765+0000 7f28f5732980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_dests= new_dests= 2024-03-04T18:06:44.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.765+0000 7f28f5732980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential dests added=[] removed=[] 2024-03-04T18:06:44.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:44.769+0000 7f28f5732980 -1 Injecting delay=5.000000000s at location=do_reshard 2024-03-04T18:06:45.622 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:45.622 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:49.769 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:49.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:49.771 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.0 2024-03-04T18:06:49.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.0 2024-03-04T18:06:49.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.1 2024-03-04T18:06:49.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.1 2024-03-04T18:06:49.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.2 2024-03-04T18:06:49.772 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.773+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.2 2024-03-04T18:06:49.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.777+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.3 2024-03-04T18:06:49.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.777+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.3 2024-03-04T18:06:49.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.777+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.4 2024-03-04T18:06:49.774 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.777+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.4 2024-03-04T18:06:49.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.777+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.5 2024-03-04T18:06:49.775 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.777+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.5 2024-03-04T18:06:49.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.781+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.6 2024-03-04T18:06:49.777 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.781+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.6 2024-03-04T18:06:49.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.781+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.7 2024-03-04T18:06:49.778 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.781+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.7 2024-03-04T18:06:49.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.781+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.8 2024-03-04T18:06:49.779 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.781+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.8 2024-03-04T18:06:49.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.785+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.9 2024-03-04T18:06:49.781 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.785+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.9 2024-03-04T18:06:49.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.785+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.10 2024-03-04T18:06:49.782 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.785+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.10 2024-03-04T18:06:49.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.785+0000 7f28f5732980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.11 2024-03-04T18:06:49.784 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.785+0000 7f28f5732980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.1.11 2024-03-04T18:06:49.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.789+0000 7f28f5732980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:49.785 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.789+0000 7f28f5732980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 2024-03-04T18:06:49.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.793+0000 7f28f5732980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:49.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.793+0000 7f28f5732980 10 moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:49.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.793+0000 7f28f5732980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:49.790 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.793+0000 7f28f5732980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18, ofs0, ns] 2024-03-04T18:06:49.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.793+0000 7f27d67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595115 cookie 94430016869888 notifier 6764 bl.length()=751 2024-03-04T18:06:49.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.793+0000 7f27d67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:49.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.793+0000 7f27d67fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:49.791 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.793+0000 7f27d67fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:49.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.797+0000 7f28f5732980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_sources= new_sources= 2024-03-04T18:06:49.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.797+0000 7f28f5732980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential sources added=[] removed=[] 2024-03-04T18:06:49.795 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.797+0000 7f28f5732980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_dests= new_dests= 2024-03-04T18:06:49.796 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.797+0000 7f28f5732980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential dests added=[] removed=[] 2024-03-04T18:06:49.802 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.805+0000 7f28f5732980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-03-04T18:06:49.807 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.809+0000 7f27a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:49.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.817+0000 7f28f5732980 20 remove_watcher() i=0 2024-03-04T18:06:49.813 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.817+0000 7f28f5732980 2 removed watcher, disabling cache 2024-03-04T18:06:49.815 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.817+0000 7f28f5732980 20 remove_watcher() i=1 2024-03-04T18:06:49.818 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.821+0000 7f28f5732980 20 remove_watcher() i=2 2024-03-04T18:06:49.821 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.825+0000 7f28f5732980 20 remove_watcher() i=3 2024-03-04T18:06:49.824 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.825+0000 7f28f5732980 20 remove_watcher() i=4 2024-03-04T18:06:49.827 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.829+0000 7f28f5732980 20 remove_watcher() i=5 2024-03-04T18:06:49.830 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.833+0000 7f28f5732980 20 remove_watcher() i=6 2024-03-04T18:06:49.833 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:49.837+0000 7f28f5732980 20 remove_watcher() i=7 2024-03-04T18:06:50.844 INFO:tasks.workunit.client.0.smithi047.stderr:put object successful 2024-03-04T18:06:50.844 INFO:tasks.workunit.client.0.smithi047.stderr:TEST: check that bucket stats are correct after reshard with unlinked entries 2024-03-04T18:06:50.844 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-04T18:06:50.849 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:50.849 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:50.991 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:50.991 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:51.142 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:51.142 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:51.276 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:51.276 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:51.344 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:51.344 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:51.472 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:51.472 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:51.537 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:51.537 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:51.670 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:51.670 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:51.733 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-03-04T18:06:52.178 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-03-04T18:06:53.610 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:53.610 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:53.754 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T18:06:53.755 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket myver --object x 2024-03-04T18:06:53.790 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:53.790 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:54.051 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.053+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.052 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.053+0000 7f0c61ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 realm 2024-03-04T18:06:54.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:54.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.053 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:54.054 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:54.055 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.057+0000 7f0c61ffd980 20 searching for the correct realm 2024-03-04T18:06:54.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.061+0000 7f0c5fb15640 0 -- 172.21.15.47:0/992222369 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f0c50041040 msgr2=0x7f0c50033420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:54.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.273+0000 7f0c5fb15640 0 -- 172.21.15.47:0/992222369 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f0c580ff270 msgr2=0x7f0c580d5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:54.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:54.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:54.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:54.490 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:54.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:54.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:54.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.491 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:54.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:54.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 zone default found 2024-03-04T18:06:54.492 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 4 Realm: () 2024-03-04T18:06:54.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:54.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:54.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:54.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.493+0000 7f0c61ffd980 20 zonegroup default 2024-03-04T18:06:54.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.497+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.497+0000 7f0c61ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.493 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.497+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.494 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.497+0000 7f0c61ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.495 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.497+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.497+0000 7f0c61ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.497+0000 7f0c61ffd980 20 started sync module instance, tier type = 2024-03-04T18:06:54.496 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.497+0000 7f0c61ffd980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:54.519 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.521+0000 7f0c61ffd980 20 add_watcher() i=0 2024-03-04T18:06:54.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.521+0000 7f0c61ffd980 20 add_watcher() i=1 2024-03-04T18:06:54.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.521+0000 7f0c61ffd980 20 add_watcher() i=2 2024-03-04T18:06:54.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.521+0000 7f0c61ffd980 20 add_watcher() i=3 2024-03-04T18:06:54.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.521+0000 7f0c61ffd980 20 add_watcher() i=4 2024-03-04T18:06:54.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.521+0000 7f0c61ffd980 20 add_watcher() i=5 2024-03-04T18:06:54.520 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.521+0000 7f0c61ffd980 20 add_watcher() i=6 2024-03-04T18:06:54.522 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.525+0000 7f0c61ffd980 20 add_watcher() i=7 2024-03-04T18:06:54.523 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.525+0000 7f0c61ffd980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:54.527 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.529+0000 7f0b15fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:54.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.529+0000 7f0c61ffd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:54.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.529+0000 7f0c61ffd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:54.528 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.529+0000 7f0c61ffd980 5 note: GC not initialized 2024-03-04T18:06:54.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.529+0000 7f0b14fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:54.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:54.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 Filter name: none 2024-03-04T18:06:54.529 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0a8e7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:54.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.530 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:54.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:54.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:54.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-03-04T18:06:54.531 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 INFO: main: starting shard=0 2024-03-04T18:06:54.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:54.532 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.533+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:54.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 INFO: main: starting shard=1 2024-03-04T18:06:54.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:54.533 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:54.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 INFO: main: starting shard=2 2024-03-04T18:06:54.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:54.534 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:54.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 INFO: main: starting shard=3 2024-03-04T18:06:54.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:54.535 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:54.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 INFO: main: starting shard=4 2024-03-04T18:06:54.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:54.536 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.537+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:54.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.537 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 20 INFO: main: starting shard=5 2024-03-04T18:06:54.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:54.538 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:54.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T18:06:54.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 20 INFO: main: starting shard=6 2024-03-04T18:06:54.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:54.539 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:54.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 20 INFO: main: starting shard=7 2024-03-04T18:06:54.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:54.540 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.541+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:54.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 20 INFO: main: starting shard=8 2024-03-04T18:06:54.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:54.542 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:54.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 20 INFO: main: starting shard=9 2024-03-04T18:06:54.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:54.543 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:54.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 20 INFO: main: starting shard=10 2024-03-04T18:06:54.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:54.544 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.545+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:54.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 20 INFO: main: starting shard=11 2024-03-04T18:06:54.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:54.545 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:54.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 20 INFO: main: starting shard=12 2024-03-04T18:06:54.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:54.546 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:54.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:54.547 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.549+0000 7f0c61ffd980 20 INFO: main: done 2024-03-04T18:06:54.549 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.553+0000 7f0b15fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:54.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.557+0000 7f0c61ffd980 20 remove_watcher() i=0 2024-03-04T18:06:54.554 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.557+0000 7f0c61ffd980 2 removed watcher, disabling cache 2024-03-04T18:06:54.556 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.557+0000 7f0c61ffd980 20 remove_watcher() i=1 2024-03-04T18:06:54.558 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.561+0000 7f0c61ffd980 20 remove_watcher() i=2 2024-03-04T18:06:54.561 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.565+0000 7f0c61ffd980 20 remove_watcher() i=3 2024-03-04T18:06:54.563 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.565+0000 7f0c61ffd980 20 remove_watcher() i=4 2024-03-04T18:06:54.566 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.569+0000 7f0c61ffd980 20 remove_watcher() i=5 2024-03-04T18:06:54.569 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.573+0000 7f0c61ffd980 20 remove_watcher() i=6 2024-03-04T18:06:54.572 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.573+0000 7f0c61ffd980 20 remove_watcher() i=7 2024-03-04T18:06:54.593 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:54.593 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:54.630 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:54.630 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:54.773 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T18:06:54.774 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket myver --object y 2024-03-04T18:06:54.810 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:54.810 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:54.855 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.857+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 realm 2024-03-04T18:06:54.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.857 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:54.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:54.858 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:54.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:54.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:54.859 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.861+0000 7f2a8c754980 20 searching for the correct realm 2024-03-04T18:06:54.868 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:54.869+0000 7f2a8aa6d640 0 -- 172.21.15.47:0/2829812582 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x56223d648970 msgr2=0x56223d64adc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:55.075 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.077+0000 7f2a8aa6d640 0 -- 172.21.15.47:0/2829812582 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x7f2954075c60 msgr2=0x7f2954078090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:55.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:55.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:55.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:55.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:55.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:55.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:55.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:55.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:55.092 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.093+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 zone default found 2024-03-04T18:06:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 4 Realm: () 2024-03-04T18:06:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:55.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:55.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 zonegroup default 2024-03-04T18:06:55.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:55.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:55.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 started sync module instance, tier type = 2024-03-04T18:06:55.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.097+0000 7f2a8c754980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:55.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.121+0000 7f2a8c754980 20 add_watcher() i=0 2024-03-04T18:06:55.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.121+0000 7f2a8c754980 20 add_watcher() i=1 2024-03-04T18:06:55.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.121+0000 7f2a8c754980 20 add_watcher() i=2 2024-03-04T18:06:55.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.121+0000 7f2a8c754980 20 add_watcher() i=3 2024-03-04T18:06:55.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.121+0000 7f2a8c754980 20 add_watcher() i=4 2024-03-04T18:06:55.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.121+0000 7f2a8c754980 20 add_watcher() i=5 2024-03-04T18:06:55.120 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.121+0000 7f2a8c754980 20 add_watcher() i=6 2024-03-04T18:06:55.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.125+0000 7f2a8c754980 20 add_watcher() i=7 2024-03-04T18:06:55.122 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.125+0000 7f2a8c754980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:55.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.129+0000 7f2940fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:55.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.129+0000 7f2a8c754980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:55.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.129+0000 7f2a8c754980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:55.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.129+0000 7f2a8c754980 5 note: GC not initialized 2024-03-04T18:06:55.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.129+0000 7f28bb7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:55.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:55.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 Filter name: none 2024-03-04T18:06:55.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f28b97fa640 20 reqs_thread_entry: start 2024-03-04T18:06:55.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:55.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-03-04T18:06:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 INFO: main: starting shard=0 2024-03-04T18:06:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:55.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:55.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 INFO: main: starting shard=1 2024-03-04T18:06:55.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:55.132 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.133+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:55.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 20 INFO: main: starting shard=2 2024-03-04T18:06:55.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:55.133 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:55.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 20 INFO: main: starting shard=3 2024-03-04T18:06:55.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:55.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:55.135 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 20 INFO: main: starting shard=4 2024-03-04T18:06:55.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:55.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.137+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:55.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.136 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 INFO: main: starting shard=5 2024-03-04T18:06:55.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:55.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:55.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 INFO: main: starting shard=6 2024-03-04T18:06:55.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:55.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:55.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 INFO: main: starting shard=7 2024-03-04T18:06:55.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:55.139 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:55.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 INFO: main: starting shard=8 2024-03-04T18:06:55.140 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:55.141 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.141+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:55.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 20 INFO: main: starting shard=9 2024-03-04T18:06:55.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:55.142 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:55.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T18:06:55.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 20 INFO: main: starting shard=10 2024-03-04T18:06:55.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:55.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:55.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 20 INFO: main: starting shard=11 2024-03-04T18:06:55.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:55.144 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.145+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:55.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.149+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.149+0000 7f2a8c754980 20 INFO: main: starting shard=12 2024-03-04T18:06:55.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.149+0000 7f2a8c754980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:55.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.149+0000 7f2a8c754980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:55.145 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.149+0000 7f2a8c754980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:55.146 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.149+0000 7f2a8c754980 20 INFO: main: done 2024-03-04T18:06:55.147 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.149+0000 7f2940fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:55.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.153+0000 7f2a8c754980 20 remove_watcher() i=0 2024-03-04T18:06:55.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.153+0000 7f2a8c754980 2 removed watcher, disabling cache 2024-03-04T18:06:55.154 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.157+0000 7f2a8c754980 20 remove_watcher() i=1 2024-03-04T18:06:55.156 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.157+0000 7f2a8c754980 20 remove_watcher() i=2 2024-03-04T18:06:55.159 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.161+0000 7f2a8c754980 20 remove_watcher() i=3 2024-03-04T18:06:55.161 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.165+0000 7f2a8c754980 20 remove_watcher() i=4 2024-03-04T18:06:55.164 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.165+0000 7f2a8c754980 20 remove_watcher() i=5 2024-03-04T18:06:55.166 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.169+0000 7f2a8c754980 20 remove_watcher() i=6 2024-03-04T18:06:55.169 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.173+0000 7f2a8c754980 20 remove_watcher() i=7 2024-03-04T18:06:55.188 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:55.188 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:55.215 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:55.215 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:55.321 INFO:tasks.workunit.client.0.smithi047.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T18:06:55.322 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bi list --bucket myver --object z 2024-03-04T18:06:55.358 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:55.358 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:55.589 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.593+0000 7f062c278640 0 --2- 172.21.15.47:0/590439780 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55afb352cbf0 0x55afb36211f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-03-04T18:06:55.615 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.617+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:55.618 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 20 realm 2024-03-04T18:06:55.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:55.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:55.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:55.619 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:55.620 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.621+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:55.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.625+0000 7f062ef61980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:55.621 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.625+0000 7f062ef61980 20 searching for the correct realm 2024-03-04T18:06:55.625 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.629+0000 7f062ca79640 0 -- 172.21.15.47:0/3487504756 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f061c04f040 msgr2=0x7f061c04f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:55.840 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:55.841+0000 7f062ca79640 0 -- 172.21.15.47:0/3487504756 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f062407cc20 msgr2=0x7f0624071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:56.057 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:56.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:56.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:56.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:56.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:56.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:56.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:56.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:56.058 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:56.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:56.059 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:56.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:56.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 zone default found 2024-03-04T18:06:56.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 4 Realm: () 2024-03-04T18:06:56.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:56.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:56.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:56.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 zonegroup default 2024-03-04T18:06:56.060 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.061+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:56.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.065+0000 7f062ef61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:56.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.065+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:56.061 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.065+0000 7f062ef61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:56.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.065+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:56.062 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.065+0000 7f062ef61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:56.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.065+0000 7f062ef61980 20 started sync module instance, tier type = 2024-03-04T18:06:56.063 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.065+0000 7f062ef61980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:56.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.089+0000 7f062ef61980 20 add_watcher() i=0 2024-03-04T18:06:56.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.089+0000 7f062ef61980 20 add_watcher() i=1 2024-03-04T18:06:56.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.089+0000 7f062ef61980 20 add_watcher() i=2 2024-03-04T18:06:56.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.089+0000 7f062ef61980 20 add_watcher() i=3 2024-03-04T18:06:56.087 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.089+0000 7f062ef61980 20 add_watcher() i=4 2024-03-04T18:06:56.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.089+0000 7f062ef61980 20 add_watcher() i=5 2024-03-04T18:06:56.088 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.089+0000 7f062ef61980 20 add_watcher() i=6 2024-03-04T18:06:56.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.093+0000 7f062ef61980 20 add_watcher() i=7 2024-03-04T18:06:56.091 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.093+0000 7f062ef61980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:56.093 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.097+0000 7f04e37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:56.094 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.097+0000 7f062ef61980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:56.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.097+0000 7f062ef61980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:56.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.097+0000 7f062ef61980 5 note: GC not initialized 2024-03-04T18:06:56.095 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.097+0000 7f04e27e4640 20 reqs_thread_entry: start 2024-03-04T18:06:56.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.097+0000 7f062ef61980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:56.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.097+0000 7f062ef61980 20 Filter name: none 2024-03-04T18:06:56.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.097+0000 7f04db7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:56.096 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.097+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:56.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:56.097 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 rados->read ofs=0 len=0 2024-03-04T18:06:56.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:56.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:56.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-03-04T18:06:56.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 INFO: main: starting shard=0 2024-03-04T18:06:56.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:56.098 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:56.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T18:06:56.099 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 INFO: main: starting shard=1 2024-03-04T18:06:56.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:56.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:56.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 INFO: main: starting shard=2 2024-03-04T18:06:56.100 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:56.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.101+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:56.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.101 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 20 INFO: main: starting shard=3 2024-03-04T18:06:56.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:56.102 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:56.103 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.103 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 20 INFO: main: starting shard=4 2024-03-04T18:06:56.103 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:56.103 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:56.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 20 INFO: main: starting shard=5 2024-03-04T18:06:56.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:56.104 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.105+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:56.105 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.109+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.109+0000 7f062ef61980 20 INFO: main: starting shard=6 2024-03-04T18:06:56.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.109+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:56.106 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.109+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:56.107 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.109+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.109+0000 7f062ef61980 20 INFO: main: starting shard=7 2024-03-04T18:06:56.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.109+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:56.108 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.109+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:56.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 INFO: main: starting shard=8 2024-03-04T18:06:56.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:56.109 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:56.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 INFO: main: starting shard=9 2024-03-04T18:06:56.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:56.110 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:56.111 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 INFO: main: starting shard=10 2024-03-04T18:06:56.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:56.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:56.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 INFO: main: starting shard=11 2024-03-04T18:06:56.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:56.112 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.113+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:56.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.117+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.117+0000 7f062ef61980 20 INFO: main: starting shard=12 2024-03-04T18:06:56.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.117+0000 7f062ef61980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:56.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.117+0000 7f062ef61980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:56.113 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.117+0000 7f062ef61980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:56.114 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.117+0000 7f062ef61980 20 INFO: main: done 2024-03-04T18:06:56.115 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.117+0000 7f04e37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:56.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.121+0000 7f062ef61980 20 remove_watcher() i=0 2024-03-04T18:06:56.119 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.121+0000 7f062ef61980 2 removed watcher, disabling cache 2024-03-04T18:06:56.121 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.125+0000 7f062ef61980 20 remove_watcher() i=1 2024-03-04T18:06:56.124 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.125+0000 7f062ef61980 20 remove_watcher() i=2 2024-03-04T18:06:56.126 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.129+0000 7f062ef61980 20 remove_watcher() i=3 2024-03-04T18:06:56.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.133+0000 7f062ef61980 20 remove_watcher() i=4 2024-03-04T18:06:56.131 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.133+0000 7f062ef61980 20 remove_watcher() i=5 2024-03-04T18:06:56.134 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.137+0000 7f062ef61980 20 remove_watcher() i=6 2024-03-04T18:06:56.137 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:56.141+0000 7f062ef61980 20 remove_watcher() i=7 2024-03-04T18:06:56.156 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:56.156 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:56.184 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-03-04T18:06:56.616 INFO:tasks.workunit.client.0.smithi047.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-03-04T18:06:57.050 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 17 --yes-i-really-mean-it 2024-03-04T18:06:57.084 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:57.084 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:57.125 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.129+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.127 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.129+0000 7fa770e14980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.129+0000 7fa770e14980 20 realm 2024-03-04T18:06:57.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.129+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.129+0000 7fa770e14980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.128 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.129+0000 7fa770e14980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:57.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.129+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.129+0000 7fa770e14980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.129+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.133+0000 7fa770e14980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:57.129 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.133+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.133+0000 7fa770e14980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:57.130 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.133+0000 7fa770e14980 20 searching for the correct realm 2024-03-04T18:06:57.138 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.141+0000 7fa76e92c640 0 -- 172.21.15.47:0/2725316175 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fa7600c5040 msgr2=0x7fa7600c5420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:57.143 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.145+0000 7fa76e92c640 0 -- 172.21.15.47:0/2725316175 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x557e40817770 msgr2=0x557e40819b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:57.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:57.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:57.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:57.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:57.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:57.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:57.150 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.151 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 zone default found 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 4 Realm: () 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 zonegroup default 2024-03-04T18:06:57.152 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.153+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.157+0000 7fa770e14980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.157+0000 7fa770e14980 20 started sync module instance, tier type = 2024-03-04T18:06:57.153 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.157+0000 7fa770e14980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:57.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.173+0000 7fa770e14980 20 add_watcher() i=0 2024-03-04T18:06:57.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.173+0000 7fa770e14980 20 add_watcher() i=1 2024-03-04T18:06:57.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.173+0000 7fa770e14980 20 add_watcher() i=2 2024-03-04T18:06:57.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.173+0000 7fa770e14980 20 add_watcher() i=3 2024-03-04T18:06:57.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.173+0000 7fa770e14980 20 add_watcher() i=4 2024-03-04T18:06:57.171 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.173+0000 7fa770e14980 20 add_watcher() i=5 2024-03-04T18:06:57.172 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.173+0000 7fa770e14980 20 add_watcher() i=6 2024-03-04T18:06:57.173 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.177+0000 7fa770e14980 20 add_watcher() i=7 2024-03-04T18:06:57.174 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.177+0000 7fa770e14980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:57.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa625fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:57.177 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:57.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:57.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 5 note: GC not initialized 2024-03-04T18:06:57.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa624fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:57.178 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:57.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 20 Filter name: none 2024-03-04T18:06:57.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa61dffb640 20 reqs_thread_entry: start 2024-03-04T18:06:57.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-03-04T18:06:57.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:57.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-03-04T18:06:57.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-03-04T18:06:57.179 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 : miss 2024-03-04T18:06:57.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:57.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:57.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 10 adding default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:57.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:57.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:57.180 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.181+0000 7fa770e14980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 2024-03-04T18:06:57.190 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.193+0000 7fa770e14980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:57.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.193+0000 7fa770e14980 10 moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:57.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.193+0000 7fa770e14980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:57.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.193+0000 7fa770e14980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18, ofs0, ns] 2024-03-04T18:06:57.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.193+0000 7fa6527fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595116 cookie 94000736505792 notifier 6857 bl.length()=783 2024-03-04T18:06:57.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.193+0000 7fa6527fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:57.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.193+0000 7fa6527fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:57.191 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.193+0000 7fa6527fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:57.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.197+0000 7fa770e14980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_sources= new_sources= 2024-03-04T18:06:57.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.197+0000 7fa770e14980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential sources added=[] removed=[] 2024-03-04T18:06:57.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.197+0000 7fa770e14980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_dests= new_dests= 2024-03-04T18:06:57.194 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.197+0000 7fa770e14980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential dests added=[] removed=[] 2024-03-04T18:06:57.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.0 2024-03-04T18:06:57.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.0 2024-03-04T18:06:57.202 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.1 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.1 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.2 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.2 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.3 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.3 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.4 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.4 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.5 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.5 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.6 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.6 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.7 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.7 2024-03-04T18:06:57.203 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.8 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.8 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.10 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.10 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.11 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.11 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.12 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.12 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.13 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.13 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.14 2024-03-04T18:06:57.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.14 2024-03-04T18:06:57.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.15 2024-03-04T18:06:57.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.15 2024-03-04T18:06:57.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.16 2024-03-04T18:06:57.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.16 2024-03-04T18:06:57.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:57.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.0 2024-03-04T18:06:57.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:57.205 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.205+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.1 2024-03-04T18:06:57.206 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.209+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:57.206 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.209+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.2 2024-03-04T18:06:57.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.209+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:57.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.209+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.3 2024-03-04T18:06:57.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.209+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:57.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.209+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.4 2024-03-04T18:06:57.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.213+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:57.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.213+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.5 2024-03-04T18:06:57.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.213+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:57.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.213+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.6 2024-03-04T18:06:57.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.213+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:57.212 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.213+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.7 2024-03-04T18:06:57.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.217+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:57.213 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.217+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.8 2024-03-04T18:06:57.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.217+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:57.215 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.217+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.9 2024-03-04T18:06:57.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.217+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:57.216 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.217+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.10 2024-03-04T18:06:57.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.221+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:57.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.221+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.11 2024-03-04T18:06:57.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.221+0000 7fa770e14980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:57.217 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.221+0000 7fa770e14980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.2.12 2024-03-04T18:06:57.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa770e14980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:57.220 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa770e14980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 2024-03-04T18:06:57.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa770e14980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:57.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa770e14980 10 moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:57.222 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa770e14980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:57.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa770e14980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18, ofs0, ns] 2024-03-04T18:06:57.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa6527fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595117 cookie 94000736505792 notifier 6857 bl.length()=751 2024-03-04T18:06:57.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa6527fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:57.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa6527fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:57.223 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.225+0000 7fa6527fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:57.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.229+0000 7fa770e14980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_sources= new_sources= 2024-03-04T18:06:57.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.229+0000 7fa770e14980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential sources added=[] removed=[] 2024-03-04T18:06:57.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.229+0000 7fa770e14980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): orig_dests= new_dests= 2024-03-04T18:06:57.226 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.229+0000 7fa770e14980 20 handle_bi_update(): bucket=:myver[797b67a4-b876-4d04-b122-10afd858d465.4453.18]): potential dests added=[] removed=[] 2024-03-04T18:06:57.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.237+0000 7fa770e14980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-03-04T18:06:57.238 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.241+0000 7fa625fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:57.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.245+0000 7fa770e14980 20 remove_watcher() i=0 2024-03-04T18:06:57.242 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.245+0000 7fa770e14980 2 removed watcher, disabling cache 2024-03-04T18:06:57.244 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.245+0000 7fa770e14980 20 remove_watcher() i=1 2024-03-04T18:06:57.247 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.249+0000 7fa770e14980 20 remove_watcher() i=2 2024-03-04T18:06:57.249 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.253+0000 7fa770e14980 20 remove_watcher() i=3 2024-03-04T18:06:57.251 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.253+0000 7fa770e14980 20 remove_watcher() i=4 2024-03-04T18:06:57.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.257+0000 7fa770e14980 20 remove_watcher() i=5 2024-03-04T18:06:57.256 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.257+0000 7fa770e14980 20 remove_watcher() i=6 2024-03-04T18:06:57.258 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.261+0000 7fa770e14980 20 remove_watcher() i=7 2024-03-04T18:06:57.281 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket check unlinked --bucket myver --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-03-04T18:06:57.321 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:57.321 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:57.365 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.369+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.369+0000 7fdbe816f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.367 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.369+0000 7fdbe816f980 20 realm 2024-03-04T18:06:57.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.369+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.369+0000 7fdbe816f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.369+0000 7fdbe816f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:57.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.369+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.368 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.369+0000 7fdbe816f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.369+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.373+0000 7fdbe816f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:57.369 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.373+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.373+0000 7fdbe816f980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:57.370 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.373+0000 7fdbe816f980 20 searching for the correct realm 2024-03-04T18:06:57.374 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.377+0000 7fdbe5486640 0 -- 172.21.15.47:0/1811671323 >> [v2:172.21.15.152:6808/929675582,v1:172.21.15.152:6809/929675582] conn(0x55f89bceab30 msgr2=0x55f89bcecfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:57.382 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.385+0000 7fdbe5c87640 0 -- 172.21.15.47:0/1811671323 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fdbe0032030 msgr2=0x7fdbe00811e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:57.601 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:57.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:57.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:57.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:57.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:57.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:57.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.602 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:57.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:57.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 zone default found 2024-03-04T18:06:57.603 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 4 Realm: () 2024-03-04T18:06:57.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:57.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:57.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:57.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 zonegroup default 2024-03-04T18:06:57.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.604 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.605+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.609+0000 7fdbe816f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.605 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.609+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.609+0000 7fdbe816f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:57.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.609+0000 7fdbe816f980 20 started sync module instance, tier type = 2024-03-04T18:06:57.606 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.609+0000 7fdbe816f980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:57.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.633+0000 7fdbe816f980 20 add_watcher() i=0 2024-03-04T18:06:57.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.633+0000 7fdbe816f980 20 add_watcher() i=1 2024-03-04T18:06:57.630 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.633+0000 7fdbe816f980 20 add_watcher() i=2 2024-03-04T18:06:57.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.633+0000 7fdbe816f980 20 add_watcher() i=3 2024-03-04T18:06:57.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.633+0000 7fdbe816f980 20 add_watcher() i=4 2024-03-04T18:06:57.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.633+0000 7fdbe816f980 20 add_watcher() i=5 2024-03-04T18:06:57.631 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.633+0000 7fdbe816f980 20 add_watcher() i=6 2024-03-04T18:06:57.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.637+0000 7fdbe816f980 20 add_watcher() i=7 2024-03-04T18:06:57.633 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.637+0000 7fdbe816f980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:57.637 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fda977fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:57.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fdbe816f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:57.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fdbe816f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:57.638 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fdbe816f980 5 note: GC not initialized 2024-03-04T18:06:57.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fda967fc640 20 reqs_thread_entry: start 2024-03-04T18:06:57.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fdbe816f980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:57.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fdbe816f980 20 Filter name: none 2024-03-04T18:06:57.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fda07fff640 20 reqs_thread_entry: start 2024-03-04T18:06:57.639 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fdbe816f980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-03-04T18:06:57.640 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.641+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:57.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-03-04T18:06:57.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-03-04T18:06:57.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 : miss 2024-03-04T18:06:57.641 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 20 rados->read ofs=0 len=0 2024-03-04T18:06:57.642 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:57.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 info.flags=0x17 2024-03-04T18:06:57.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 adding default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 to cache LRU end 2024-03-04T18:06:57.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:57.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-03-04T18:06:57.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:797b67a4-b876-4d04-b122-10afd858d465.4453.18 2024-03-04T18:06:57.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.0 2024-03-04T18:06:57.643 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.0 2024-03-04T18:06:57.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 1 NOTICE: finished shard 0 (0 entries removed) 2024-03-04T18:06:57.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.1 2024-03-04T18:06:57.644 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.645+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.1 2024-03-04T18:06:57.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.649+0000 7fdbe816f980 1 NOTICE: finished shard 1 (0 entries removed) 2024-03-04T18:06:57.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.649+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.2 2024-03-04T18:06:57.645 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.649+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.2 2024-03-04T18:06:57.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.649+0000 7fdbe816f980 1 NOTICE: finished shard 2 (0 entries removed) 2024-03-04T18:06:57.646 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.649+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.3 2024-03-04T18:06:57.647 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.649+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.3 2024-03-04T18:06:57.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.649+0000 7fdbe816f980 1 NOTICE: finished shard 3 (0 entries removed) 2024-03-04T18:06:57.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.649+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.4 2024-03-04T18:06:57.648 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.649+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.4 2024-03-04T18:06:57.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.653+0000 7fdbe816f980 1 NOTICE: finished shard 4 (0 entries removed) 2024-03-04T18:06:57.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.653+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.5 2024-03-04T18:06:57.649 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.653+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.5 2024-03-04T18:06:57.650 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.653+0000 7fdbe816f980 1 NOTICE: finished shard 5 (0 entries removed) 2024-03-04T18:06:57.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.653+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.6 2024-03-04T18:06:57.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.653+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.6 2024-03-04T18:06:57.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.653+0000 7fdbe816f980 1 NOTICE: finished shard 6 (0 entries removed) 2024-03-04T18:06:57.651 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.653+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.7 2024-03-04T18:06:57.652 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.653+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.7 2024-03-04T18:06:57.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 1 NOTICE: finished shard 7 (0 entries removed) 2024-03-04T18:06:57.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.8 2024-03-04T18:06:57.653 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.8 2024-03-04T18:06:57.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 1 NOTICE: finished shard 8 (0 entries removed) 2024-03-04T18:06:57.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.654 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.10 2024-03-04T18:06:57.655 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.10 2024-03-04T18:06:57.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 1 NOTICE: finished shard 10 (0 entries removed) 2024-03-04T18:06:57.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.11 2024-03-04T18:06:57.656 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.657+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.11 2024-03-04T18:06:57.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 1 NOTICE: finished shard 11 (0 entries removed) 2024-03-04T18:06:57.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.12 2024-03-04T18:06:57.657 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.12 2024-03-04T18:06:57.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 1 NOTICE: finished shard 12 (0 entries removed) 2024-03-04T18:06:57.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.13 2024-03-04T18:06:57.658 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.13 2024-03-04T18:06:57.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 1 NOTICE: finished shard 13 (0 entries removed) 2024-03-04T18:06:57.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.14 2024-03-04T18:06:57.659 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.14 2024-03-04T18:06:57.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 1 NOTICE: finished shard 14 (0 entries removed) 2024-03-04T18:06:57.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.15 2024-03-04T18:06:57.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.661+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.15 2024-03-04T18:06:57.660 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.665+0000 7fdbe816f980 1 NOTICE: finished shard 15 (0 entries removed) 2024-03-04T18:06:57.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.665+0000 7fdbe816f980 10 bucket_obj is .dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.16 2024-03-04T18:06:57.661 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.665+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.16 2024-03-04T18:06:57.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.665+0000 7fdbe816f980 1 NOTICE: finished shard 16 (0 entries removed) 2024-03-04T18:06:57.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.665+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.662 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.665+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:x state=0x55f89bd1b558 s->prefetch_data=0 2024-03-04T18:06:57.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.665+0000 7fdbe816f980 20 get_obj_state: setting s->obj_tag to jmk0rgm4tokdcyq7rxreniai1a9fbxpw 2024-03-04T18:06:57.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.665+0000 7fdbe816f980 20 get_obj_state_impl: setting s->olh_tag to k9dneereqt7w9oa03txlc3d1lgta2kdc 2024-03-04T18:06:57.663 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.665+0000 7fdbe816f980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=k9dneereqt7w9oa03txlc3d1lgta2kdc 2024-03-04T18:06:57.666 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.669+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.669 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.673+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.669 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.673+0000 7fdbe816f980 20 olh_log_entry: epoch=2 op=2 key=x[] 2024-03-04T18:06:57.669 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.673+0000 7fdbe816f980 20 olh_log_entry: epoch=2 op=3 key=x[7lZea8Uxh2o958yKRqLU1y-cr53eN9E] 2024-03-04T18:06:57.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.673+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:_:7lZea8Uxh2o958yKRqLU1y-cr53eN9E_x state=0x55f89bd22418 s->prefetch_data=0 2024-03-04T18:06:57.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.673+0000 7fdbe816f980 10 manifest: total_size = 11 2024-03-04T18:06:57.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.673+0000 7fdbe816f980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.16497541892555045705 2024-03-04T18:06:57.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.673+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:_:7lZea8Uxh2o958yKRqLU1y-cr53eN9E_x state=0x55f89bd22418 s->prefetch_data=0 2024-03-04T18:06:57.670 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.673+0000 7fdbe816f980 20 prepare_atomic_modification: state is not atomic. state=0x55f89bd22418 2024-03-04T18:06:57.671 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.673+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.679 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.681+0000 7fdac97fa640 20 handle_completion(): completion ok for obj=x[7lZea8Uxh2o958yKRqLU1y-cr53eN9E] 2024-03-04T18:06:57.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.681+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:x state=0x55f89bd1b558 s->prefetch_data=0 2024-03-04T18:06:57.680 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.681+0000 7fdbe816f980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=k9dneereqt7w9oa03txlc3d1lgta2kdc 2024-03-04T18:06:57.687 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.689+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.690 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.693+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.690 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.693+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.691 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.693+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:y state=0x55f89bd1b558 s->prefetch_data=0 2024-03-04T18:06:57.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.693+0000 7fdbe816f980 20 get_obj_state: setting s->obj_tag to wwf0z8eiq1cinmoc528h0w5yx9dord9j 2024-03-04T18:06:57.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.693+0000 7fdbe816f980 20 get_obj_state_impl: setting s->olh_tag to 5e9agrg4wd0a4aynnxiuoq58da24bzs9 2024-03-04T18:06:57.692 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.693+0000 7fdbe816f980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=5e9agrg4wd0a4aynnxiuoq58da24bzs9 2024-03-04T18:06:57.695 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.697+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.701+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.701+0000 7fdbe816f980 20 olh_log_entry: epoch=2 op=2 key=y[] 2024-03-04T18:06:57.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.701+0000 7fdbe816f980 20 olh_log_entry: epoch=2 op=3 key=y[iX07HsFu2iM34eyPvsQqZwDZLzYVdMY] 2024-03-04T18:06:57.698 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.701+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:_:iX07HsFu2iM34eyPvsQqZwDZLzYVdMY_y state=0x55f89bd22398 s->prefetch_data=0 2024-03-04T18:06:57.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.701+0000 7fdbe816f980 10 manifest: total_size = 12 2024-03-04T18:06:57.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.701+0000 7fdbe816f980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.6982693244177345335 2024-03-04T18:06:57.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.701+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:_:iX07HsFu2iM34eyPvsQqZwDZLzYVdMY_y state=0x55f89bd22398 s->prefetch_data=0 2024-03-04T18:06:57.699 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.701+0000 7fdbe816f980 20 prepare_atomic_modification: state is not atomic. state=0x55f89bd22398 2024-03-04T18:06:57.700 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.701+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.713+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:y state=0x55f89bd1b558 s->prefetch_data=0 2024-03-04T18:06:57.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.713+0000 7fdbe816f980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=5e9agrg4wd0a4aynnxiuoq58da24bzs9 2024-03-04T18:06:57.709 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.713+0000 7fdac97fa640 20 handle_completion(): completion ok for obj=y[iX07HsFu2iM34eyPvsQqZwDZLzYVdMY] 2024-03-04T18:06:57.717 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.721+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.720 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.721+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.725+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.721 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.725+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:z state=0x55f89bd51798 s->prefetch_data=0 2024-03-04T18:06:57.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.725+0000 7fdbe816f980 20 get_obj_state: setting s->obj_tag to hx38yl6k3daxcr9row58v4frgtqxqmle 2024-03-04T18:06:57.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.725+0000 7fdbe816f980 20 get_obj_state_impl: setting s->olh_tag to r60i58x54ngwft9p6jwn0qxx3l1twxoq 2024-03-04T18:06:57.722 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.725+0000 7fdbe816f980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=r60i58x54ngwft9p6jwn0qxx3l1twxoq 2024-03-04T18:06:57.727 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.729+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.729 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.733+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.733+0000 7fdbe816f980 20 olh_log_entry: epoch=2 op=2 key=z[] 2024-03-04T18:06:57.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.733+0000 7fdbe816f980 20 olh_log_entry: epoch=2 op=3 key=z[B6UboVw2RjC7Ge0FlGdUnR0xyYMJBOw] 2024-03-04T18:06:57.730 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.733+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:_:B6UboVw2RjC7Ge0FlGdUnR0xyYMJBOw_z state=0x55f89bd22398 s->prefetch_data=0 2024-03-04T18:06:57.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.733+0000 7fdbe816f980 10 manifest: total_size = 12 2024-03-04T18:06:57.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.733+0000 7fdbe816f980 20 get_obj_state: setting s->obj_tag to 797b67a4-b876-4d04-b122-10afd858d465.4453.6290802459965520313 2024-03-04T18:06:57.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.733+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:_:B6UboVw2RjC7Ge0FlGdUnR0xyYMJBOw_z state=0x55f89bd22398 s->prefetch_data=0 2024-03-04T18:06:57.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.733+0000 7fdbe816f980 20 prepare_atomic_modification: state is not atomic. state=0x55f89bd22398 2024-03-04T18:06:57.731 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.733+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.740 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.741+0000 7fdac97fa640 20 handle_completion(): completion ok for obj=z[B6UboVw2RjC7Ge0FlGdUnR0xyYMJBOw] 2024-03-04T18:06:57.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.745+0000 7fdbe816f980 20 get_obj_state: octx=0x55f89ba71950 obj=myver:z state=0x55f89bd51798 s->prefetch_data=0 2024-03-04T18:06:57.741 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.745+0000 7fdbe816f980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=r60i58x54ngwft9p6jwn0qxx3l1twxoq 2024-03-04T18:06:57.749 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.753+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.753+0000 7fdbe816f980 20 bucket index object: :.dir.797b67a4-b876-4d04-b122-10afd858d465.4453.18.3.9 2024-03-04T18:06:57.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.753+0000 7fdbe816f980 1 NOTICE: finished shard 9 (3 entries removed) 2024-03-04T18:06:57.752 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.753+0000 7fdbe816f980 1 NOTICE: finished all shards (3 entries removed) 2024-03-04T18:06:57.753 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.757+0000 7fda977fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:57.758 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.761+0000 7fdbe816f980 20 remove_watcher() i=0 2024-03-04T18:06:57.759 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.761+0000 7fdbe816f980 2 removed watcher, disabling cache 2024-03-04T18:06:57.761 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.765+0000 7fdbe816f980 20 remove_watcher() i=1 2024-03-04T18:06:57.763 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.765+0000 7fdbe816f980 20 remove_watcher() i=2 2024-03-04T18:06:57.766 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.769+0000 7fdbe816f980 20 remove_watcher() i=3 2024-03-04T18:06:57.768 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.769+0000 7fdbe816f980 20 remove_watcher() i=4 2024-03-04T18:06:57.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.773+0000 7fdbe816f980 20 remove_watcher() i=5 2024-03-04T18:06:57.773 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.777+0000 7fdbe816f980 20 remove_watcher() i=6 2024-03-04T18:06:57.776 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:57.777+0000 7fdbe816f980 20 remove_watcher() i=7 2024-03-04T18:06:57.797 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:57.797 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:57.867 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:57.867 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:58.125 INFO:tasks.workunit.client.0.smithi047.stderr:radosgw-admin bucket stats --bucket myver 2024-03-04T18:06:58.159 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:58.159 INFO:tasks.workunit.client.0.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:58.204 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.205+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.209+0000 7f3621fa8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:58.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.209+0000 7f3621fa8980 20 realm 2024-03-04T18:06:58.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.209+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.209+0000 7f3621fa8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:58.207 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.209+0000 7f3621fa8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:58.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.209+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.209+0000 7f3621fa8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:58.208 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.209+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.213+0000 7f3621fa8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:58.209 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.213+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.213+0000 7f3621fa8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:58.210 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.213+0000 7f3621fa8980 20 searching for the correct realm 2024-03-04T18:06:58.218 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.221+0000 7f36202c1640 0 -- 172.21.15.47:0/3023391764 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55b58f9a8050 msgr2=0x55b58f9aa4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:58.224 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.229+0000 7f361fac0640 0 -- 172.21.15.47:0/3023391764 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x55b58f9ac1f0 msgr2=0x55b58f9ae6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:58.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.233+0000 7f3621fa8980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:06:58.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.233+0000 7f3621fa8980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:06:58.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.233+0000 7f3621fa8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:58.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.233+0000 7f3621fa8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:58.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.233+0000 7f3621fa8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:58.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.233+0000 7f3621fa8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:58.232 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.233+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.233+0000 7f3621fa8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:58.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.233+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:58.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:58.233 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 zone default found 2024-03-04T18:06:58.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 4 Realm: () 2024-03-04T18:06:58.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:06:58.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:06:58.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:58.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 zonegroup default 2024-03-04T18:06:58.234 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:58.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:58.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:58.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 started sync module instance, tier type = 2024-03-04T18:06:58.235 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.237+0000 7f3621fa8980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:06:58.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.257+0000 7f3621fa8980 20 add_watcher() i=0 2024-03-04T18:06:58.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.257+0000 7f3621fa8980 20 add_watcher() i=1 2024-03-04T18:06:58.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.257+0000 7f3621fa8980 20 add_watcher() i=2 2024-03-04T18:06:58.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.257+0000 7f3621fa8980 20 add_watcher() i=3 2024-03-04T18:06:58.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.257+0000 7f3621fa8980 20 add_watcher() i=4 2024-03-04T18:06:58.253 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.257+0000 7f3621fa8980 20 add_watcher() i=5 2024-03-04T18:06:58.254 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.257+0000 7f3621fa8980 20 add_watcher() i=6 2024-03-04T18:06:58.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.257+0000 7f3621fa8980 20 add_watcher() i=7 2024-03-04T18:06:58.255 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.257+0000 7f3621fa8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:58.259 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.261+0000 7f34d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:58.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.261+0000 7f3621fa8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:58.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.261+0000 7f3621fa8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:58.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.261+0000 7f3621fa8980 5 note: GC not initialized 2024-03-04T18:06:58.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.261+0000 7f34d4fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:58.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.261+0000 7f3621fa8980 20 init_complete bucket index max shards: 11 2024-03-04T18:06:58.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.261+0000 7f3621fa8980 20 Filter name: none 2024-03-04T18:06:58.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.261+0000 7f344e7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:58.260 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.261+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.265+0000 7f3621fa8980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:58.261 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.265+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.262 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.265+0000 7f3621fa8980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:58.262 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.265+0000 7f3621fa8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:58.262 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.265+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.265+0000 7f3621fa8980 20 rados_obj.operate() r=0 bl.length=173 2024-03-04T18:06:58.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.265+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.265+0000 7f3621fa8980 20 rados_obj.operate() r=0 bl.length=357 2024-03-04T18:06:58.263 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.265+0000 7f3621fa8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:58.266 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.269+0000 7f3621fa8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:58.267 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.269+0000 7f3621fa8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:58.268 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.269+0000 7f34d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:58.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.273+0000 7f3621fa8980 20 remove_watcher() i=0 2024-03-04T18:06:58.272 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.273+0000 7f3621fa8980 2 removed watcher, disabling cache 2024-03-04T18:06:58.274 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.277+0000 7f3621fa8980 20 remove_watcher() i=1 2024-03-04T18:06:58.276 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.277+0000 7f3621fa8980 20 remove_watcher() i=2 2024-03-04T18:06:58.278 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.281+0000 7f3621fa8980 20 remove_watcher() i=3 2024-03-04T18:06:58.280 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.281+0000 7f3621fa8980 20 remove_watcher() i=4 2024-03-04T18:06:58.282 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.285+0000 7f3621fa8980 20 remove_watcher() i=5 2024-03-04T18:06:58.285 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.289+0000 7f3621fa8980 20 remove_watcher() i=6 2024-03-04T18:06:58.287 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-04T18:06:58.289+0000 7f3621fa8980 20 remove_watcher() i=7 2024-03-04T18:06:58.301 INFO:tasks.workunit.client.0.smithi047.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-03-04T18:06:58.301 INFO:tasks.workunit.client.0.smithi047.stderr:Deleting bucket a-bucket 2024-03-04T18:06:58.302 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:58.302 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:58.329 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:58.329 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:58.427 INFO:tasks.workunit.client.0.smithi047.stderr:Deleting bucket myver 2024-03-04T18:06:58.429 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:58.429 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:58.565 INFO:tasks.workunit.client.0.smithi047.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:58.565 INFO:tasks.workunit.client.0.smithi047.stderr: warnings.warn( 2024-03-04T18:06:59.013 INFO:tasks.workunit.client.0.smithi047.stderr:Completed resharding tests 2024-03-04T18:06:59.014 INFO:tasks.workunit.client.0.smithi047.stdout:Could not connect to the endpoint URL: "http://localhost:80/" 2024-03-04T18:06:59.014 INFO:tasks.workunit.client.0.smithi047.stdout:Could not connect to the endpoint URL: "http://localhost:8000/" 2024-03-04T18:06:59.014 INFO:tasks.workunit.client.0.smithi047.stdout:connected to https://localhost:443 2024-03-04T18:06:59.129 INFO:tasks.workunit.client.0.smithi047.stderr:+ deactivate 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ export PATH 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ unset _OLD_VIRTUAL_PATH 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' -n '' ']' 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ hash -r 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' -n '' ']' 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ unset VIRTUAL_ENV 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-03-04T18:06:59.130 INFO:tasks.workunit.client.0.smithi047.stderr:+ '[' '!' '' = nondestructive ']' 2024-03-04T18:06:59.131 INFO:tasks.workunit.client.0.smithi047.stderr:+ unset -f deactivate 2024-03-04T18:06:59.131 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo OK. 2024-03-04T18:06:59.132 INFO:tasks.workunit.client.0.smithi047.stdout:OK. 2024-03-04T18:06:59.132 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T18:06:59.132 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T18:06:59.187 INFO:tasks.workunit:Stopping ['rgw/run-reshard.sh'] on client.0... 2024-03-04T18:06:59.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T18:06:59.623 DEBUG:teuthology.parallel:result is None 2024-03-04T18:06:59.624 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T18:06:59.635 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T18:06:59.635 DEBUG:teuthology.orchestra.run.smithi047:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T18:06:59.685 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T18:06:59.686 INFO:teuthology.run_tasks:Running task s3tests-java... 2024-03-04T18:06:59.706 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2024-03-04T18:06:59.707 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2024-03-04T18:06:59.707 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2024-03-04T18:06:59.707 DEBUG:teuthology.orchestra.run.smithi047:> git clone -b ceph-master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2024-03-04T18:06:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2024-03-04T18:07:04.736 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2024-03-04T18:07:04.737 DEBUG:teuthology.orchestra.run.smithi047:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2024-03-04T18:07:04.831 INFO:teuthology.orchestra.run.smithi047.stderr:/home/ubuntu/cephtest/s3-tests-java/bootstrap.sh: missing required DEB packages. Installing via sudo. 2024-03-04T18:07:04.890 INFO:teuthology.orchestra.run.smithi047.stdout:Reading package lists... 2024-03-04T18:07:05.089 INFO:teuthology.orchestra.run.smithi047.stdout:Building dependency tree... 2024-03-04T18:07:05.090 INFO:teuthology.orchestra.run.smithi047.stdout:Reading state information... 2024-03-04T18:07:05.275 INFO:teuthology.orchestra.run.smithi047.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T18:07:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T18:07:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T18:07:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T18:07:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T18:07:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T18:07:05.280 INFO:teuthology.orchestra.run.smithi047.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T18:07:05.280 INFO:teuthology.orchestra.run.smithi047.stdout:The following additional packages will be installed: 2024-03-04T18:07:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-03-04T18:07:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-03-04T18:07:05.280 INFO:teuthology.orchestra.run.smithi047.stdout:Suggested packages: 2024-03-04T18:07:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: gvfs openjdk-8-demo openjdk-8-source visualvm libnss-mdns fonts-nanum 2024-03-04T18:07:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei 2024-03-04T18:07:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: fonts-wqy-zenhei fonts-indic 2024-03-04T18:07:05.312 INFO:teuthology.orchestra.run.smithi047.stdout:The following NEW packages will be installed: 2024-03-04T18:07:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-03-04T18:07:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-03-04T18:07:05.495 INFO:teuthology.orchestra.run.smithi047.stdout:0 upgraded, 9 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T18:07:05.496 INFO:teuthology.orchestra.run.smithi047.stdout:Need to get 46.1 MB of archives. 2024-03-04T18:07:05.496 INFO:teuthology.orchestra.run.smithi047.stdout:After this operation, 155 MB of additional disk space will be used. 2024-03-04T18:07:05.496 INFO:teuthology.orchestra.run.smithi047.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-common all 2.24.33-2ubuntu2 [125 kB] 2024-03-04T18:07:05.867 INFO:teuthology.orchestra.run.smithi047.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-0 amd64 2.24.33-2ubuntu2 [2,037 kB] 2024-03-04T18:07:06.228 INFO:teuthology.orchestra.run.smithi047.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail18 amd64 2.24.33-2ubuntu2 [15.9 kB] 2024-03-04T18:07:06.229 INFO:teuthology.orchestra.run.smithi047.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail-common amd64 2.24.33-2ubuntu2 [132 kB] 2024-03-04T18:07:06.235 INFO:teuthology.orchestra.run.smithi047.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-bin amd64 2.24.33-2ubuntu2 [7,932 B] 2024-03-04T18:07:06.236 INFO:teuthology.orchestra.run.smithi047.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre-headless amd64 8u392-ga-1~22.04 [30.8 MB] 2024-03-04T18:07:07.295 INFO:teuthology.orchestra.run.smithi047.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre amd64 8u392-ga-1~22.04 [75.5 kB] 2024-03-04T18:07:07.296 INFO:teuthology.orchestra.run.smithi047.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk-headless amd64 8u392-ga-1~22.04 [8,863 kB] 2024-03-04T18:07:07.544 INFO:teuthology.orchestra.run.smithi047.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk amd64 8u392-ga-1~22.04 [4,006 kB] 2024-03-04T18:07:07.965 INFO:teuthology.orchestra.run.smithi047.stderr:debconf: unable to initialize frontend: Dialog 2024-03-04T18:07:07.965 INFO:teuthology.orchestra.run.smithi047.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-03-04T18:07:07.966 INFO:teuthology.orchestra.run.smithi047.stderr:debconf: falling back to frontend: Readline 2024-03-04T18:07:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:debconf: unable to initialize frontend: Readline 2024-03-04T18:07:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:debconf: (This frontend requires a controlling tty.) 2024-03-04T18:07:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:debconf: falling back to frontend: Teletype 2024-03-04T18:07:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-03-04T18:07:08.019 INFO:teuthology.orchestra.run.smithi047.stdout:Fetched 46.1 MB in 2s (19.8 MB/s) 2024-03-04T18:07:08.126 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libgtk2.0-common. 2024-03-04T18:07:08.188 INFO:teuthology.orchestra.run.smithi047.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174558 files and directories currently installed.) 2024-03-04T18:07:08.193 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../0-libgtk2.0-common_2.24.33-2ubuntu2_all.deb ... 2024-03-04T18:07:08.233 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-03-04T18:07:08.637 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libgtk2.0-0:amd64. 2024-03-04T18:07:08.666 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../1-libgtk2.0-0_2.24.33-2ubuntu2_amd64.deb ... 2024-03-04T18:07:08.745 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:07:09.200 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libgail18:amd64. 2024-03-04T18:07:09.229 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../2-libgail18_2.24.33-2ubuntu2_amd64.deb ... 2024-03-04T18:07:09.266 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:07:09.519 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libgail-common:amd64. 2024-03-04T18:07:09.536 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../3-libgail-common_2.24.33-2ubuntu2_amd64.deb ... 2024-03-04T18:07:09.577 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:07:09.880 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package libgtk2.0-bin. 2024-03-04T18:07:09.908 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../4-libgtk2.0-bin_2.24.33-2ubuntu2_amd64.deb ... 2024-03-04T18:07:09.946 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-03-04T18:07:10.325 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package openjdk-8-jre-headless:amd64. 2024-03-04T18:07:10.355 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../5-openjdk-8-jre-headless_8u392-ga-1~22.04_amd64.deb ... 2024-03-04T18:07:10.402 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking openjdk-8-jre-headless:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:07:12.038 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package openjdk-8-jre:amd64. 2024-03-04T18:07:12.067 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../6-openjdk-8-jre_8u392-ga-1~22.04_amd64.deb ... 2024-03-04T18:07:12.108 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking openjdk-8-jre:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:07:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package openjdk-8-jdk-headless:amd64. 2024-03-04T18:07:12.705 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../7-openjdk-8-jdk-headless_8u392-ga-1~22.04_amd64.deb ... 2024-03-04T18:07:12.746 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking openjdk-8-jdk-headless:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:07:13.534 INFO:teuthology.orchestra.run.smithi047.stdout:Selecting previously unselected package openjdk-8-jdk:amd64. 2024-03-04T18:07:13.563 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing to unpack .../8-openjdk-8-jdk_8u392-ga-1~22.04_amd64.deb ... 2024-03-04T18:07:13.616 INFO:teuthology.orchestra.run.smithi047.stdout:Unpacking openjdk-8-jdk:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:07:14.016 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up openjdk-8-jre-headless:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:07:15.105 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/orbd to provide /usr/bin/orbd (orbd) in auto mode 2024-03-04T18:07:15.120 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/servertool to provide /usr/bin/servertool (servertool) in auto mode 2024-03-04T18:07:15.146 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/tnameserv to provide /usr/bin/tnameserv (tnameserv) in auto mode 2024-03-04T18:07:15.562 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-03-04T18:07:15.730 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:07:16.082 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up openjdk-8-jre:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:07:16.181 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/policytool to provide /usr/bin/policytool (policytool) in auto mode 2024-03-04T18:07:16.250 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up openjdk-8-jdk-headless:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:07:16.334 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/clhsdb to provide /usr/bin/clhsdb (clhsdb) in auto mode 2024-03-04T18:07:16.363 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/extcheck to provide /usr/bin/extcheck (extcheck) in auto mode 2024-03-04T18:07:16.389 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/hsdb to provide /usr/bin/hsdb (hsdb) in auto mode 2024-03-04T18:07:16.405 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/idlj to provide /usr/bin/idlj (idlj) in auto mode 2024-03-04T18:07:16.534 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javah to provide /usr/bin/javah (javah) in auto mode 2024-03-04T18:07:16.926 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jhat to provide /usr/bin/jhat (jhat) in auto mode 2024-03-04T18:07:17.053 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jsadebugd to provide /usr/bin/jsadebugd (jsadebugd) in auto mode 2024-03-04T18:07:17.163 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/native2ascii to provide /usr/bin/native2ascii (native2ascii) in auto mode 2024-03-04T18:07:17.205 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/schemagen to provide /usr/bin/schemagen (schemagen) in auto mode 2024-03-04T18:07:17.273 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsgen to provide /usr/bin/wsgen (wsgen) in auto mode 2024-03-04T18:07:17.299 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsimport to provide /usr/bin/wsimport (wsimport) in auto mode 2024-03-04T18:07:17.324 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/xjc to provide /usr/bin/xjc (xjc) in auto mode 2024-03-04T18:07:17.414 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:07:17.539 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-03-04T18:07:17.666 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up openjdk-8-jdk:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:07:17.759 INFO:teuthology.orchestra.run.smithi047.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode 2024-03-04T18:07:17.851 INFO:teuthology.orchestra.run.smithi047.stdout:Setting up libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:07:17.961 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T18:07:19.037 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T18:07:19.129 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-04T18:07:19.423 INFO:teuthology.orchestra.run.smithi047.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T18:07:19.882 INFO:teuthology.orchestra.run.smithi047.stdout: Gradle will be installed under /opt/gradle 2024-03-04T18:07:19.914 INFO:teuthology.orchestra.run.smithi047.stderr:--2024-03-04 18:07:19-- https://services.gradle.org/distributions/gradle-6.0.1-bin.zip 2024-03-04T18:07:19.919 INFO:teuthology.orchestra.run.smithi047.stderr:Resolving services.gradle.org (services.gradle.org)... 104.17.128.37, 104.17.129.37, 2606:4700::6811:8025, ... 2024-03-04T18:07:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:Connecting to services.gradle.org (services.gradle.org)|104.17.128.37|:443... connected. 2024-03-04T18:07:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2024-03-04T18:07:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:Location: https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip [following] 2024-03-04T18:07:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:--2024-03-04 18:07:19-- https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip 2024-03-04T18:07:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:Resolving github.com (github.com)... 140.82.113.3 2024-03-04T18:07:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:Connecting to github.com (github.com)|140.82.113.3|:443... connected. 2024-03-04T18:07:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:HTTP request sent, awaiting response... 302 Found 2024-03-04T18:07:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240304%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240304T180713Z&X-Amz-Expires=300&X-Amz-Signature=798a1bb8469b160ccb2c3ac90ad5068f0b160b629e5202e1187f13906a5cdec2&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream [following] 2024-03-04T18:07:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:--2024-03-04 18:07:20-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240304%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240304T180713Z&X-Amz-Expires=300&X-Amz-Signature=798a1bb8469b160ccb2c3ac90ad5068f0b160b629e5202e1187f13906a5cdec2&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream 2024-03-04T18:07:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.109.133, ... 2024-03-04T18:07:20.049 INFO:teuthology.orchestra.run.smithi047.stderr:Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected. 2024-03-04T18:07:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-04T18:07:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:Length: 95129672 (91M) [application/octet-stream] 2024-03-04T18:07:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:Saving to: 'gradle-6.0.1-bin.zip' 2024-03-04T18:07:20.173 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-03-04T18:07:20.191 INFO:teuthology.orchestra.run.smithi047.stderr: 0K .......... .......... .......... .......... .......... 0% 2.61M 35s 2024-03-04T18:07:20.202 INFO:teuthology.orchestra.run.smithi047.stderr: 50K .......... .......... .......... .......... .......... 0% 4.30M 28s 2024-03-04T18:07:20.206 INFO:teuthology.orchestra.run.smithi047.stderr: 100K .......... .......... .......... .......... .......... 0% 12.8M 21s 2024-03-04T18:07:20.210 INFO:teuthology.orchestra.run.smithi047.stderr: 150K .......... .......... .......... .......... .......... 0% 13.2M 17s 2024-03-04T18:07:20.218 INFO:teuthology.orchestra.run.smithi047.stderr: 200K .......... .......... .......... .......... .......... 0% 6.29M 17s 2024-03-04T18:07:20.219 INFO:teuthology.orchestra.run.smithi047.stderr: 250K .......... .......... .......... .......... .......... 0% 24.0M 15s 2024-03-04T18:07:20.224 INFO:teuthology.orchestra.run.smithi047.stderr: 300K .......... .......... .......... .......... .......... 0% 12.3M 14s 2024-03-04T18:07:20.224 INFO:teuthology.orchestra.run.smithi047.stderr: 350K .......... .......... .......... .......... .......... 0% 41.1M 12s 2024-03-04T18:07:20.227 INFO:teuthology.orchestra.run.smithi047.stderr: 400K .......... .......... .......... .......... .......... 0% 20.8M 11s 2024-03-04T18:07:20.228 INFO:teuthology.orchestra.run.smithi047.stderr: 450K .......... .......... .......... .......... .......... 0% 43.1M 10s 2024-03-04T18:07:20.235 INFO:teuthology.orchestra.run.smithi047.stderr: 500K .......... .......... .......... .......... .......... 0% 7.24M 11s 2024-03-04T18:07:20.237 INFO:teuthology.orchestra.run.smithi047.stderr: 550K .......... .......... .......... .......... .......... 0% 26.7M 10s 2024-03-04T18:07:20.239 INFO:teuthology.orchestra.run.smithi047.stderr: 600K .......... .......... .......... .......... .......... 0% 30.7M 9s 2024-03-04T18:07:20.242 INFO:teuthology.orchestra.run.smithi047.stderr: 650K .......... .......... .......... .......... .......... 0% 26.4M 9s 2024-03-04T18:07:20.242 INFO:teuthology.orchestra.run.smithi047.stderr: 700K .......... .......... .......... .......... .......... 0% 68.7M 8s 2024-03-04T18:07:20.242 INFO:teuthology.orchestra.run.smithi047.stderr: 750K .......... .......... .......... .......... .......... 0% 75.5M 8s 2024-03-04T18:07:20.245 INFO:teuthology.orchestra.run.smithi047.stderr: 800K .......... .......... .......... .......... .......... 0% 26.8M 8s 2024-03-04T18:07:20.245 INFO:teuthology.orchestra.run.smithi047.stderr: 850K .......... .......... .......... .......... .......... 0% 49.2M 7s 2024-03-04T18:07:20.245 INFO:teuthology.orchestra.run.smithi047.stderr: 900K .......... .......... .......... .......... .......... 1% 93.7M 7s 2024-03-04T18:07:20.246 INFO:teuthology.orchestra.run.smithi047.stderr: 950K .......... .......... .......... .......... .......... 1% 43.3M 7s 2024-03-04T18:07:20.250 INFO:teuthology.orchestra.run.smithi047.stderr: 1000K .......... .......... .......... .......... .......... 1% 11.9M 7s 2024-03-04T18:07:20.253 INFO:teuthology.orchestra.run.smithi047.stderr: 1050K .......... .......... .......... .......... .......... 1% 22.2M 7s 2024-03-04T18:07:20.256 INFO:teuthology.orchestra.run.smithi047.stderr: 1100K .......... .......... .......... .......... .......... 1% 31.5M 7s 2024-03-04T18:07:20.256 INFO:teuthology.orchestra.run.smithi047.stderr: 1150K .......... .......... .......... .......... .......... 1% 78.0M 6s 2024-03-04T18:07:20.256 INFO:teuthology.orchestra.run.smithi047.stderr: 1200K .......... .......... .......... .......... .......... 1% 62.4M 6s 2024-03-04T18:07:20.269 INFO:teuthology.orchestra.run.smithi047.stderr: 1250K .......... .......... .......... .......... .......... 1% 25.1M 6s 2024-03-04T18:07:20.269 INFO:teuthology.orchestra.run.smithi047.stderr: 1300K .......... .......... .......... .......... .......... 1% 79.7M 6s 2024-03-04T18:07:20.269 INFO:teuthology.orchestra.run.smithi047.stderr: 1350K .......... .......... .......... .......... .......... 1% 102M 6s 2024-03-04T18:07:20.269 INFO:teuthology.orchestra.run.smithi047.stderr: 1400K .......... .......... .......... .......... .......... 1% 38.7M 6s 2024-03-04T18:07:20.269 INFO:teuthology.orchestra.run.smithi047.stderr: 1450K .......... .......... .......... .......... .......... 1% 87.1M 5s 2024-03-04T18:07:20.269 INFO:teuthology.orchestra.run.smithi047.stderr: 1500K .......... .......... .......... .......... .......... 1% 49.4M 5s 2024-03-04T18:07:20.269 INFO:teuthology.orchestra.run.smithi047.stderr: 1550K .......... .......... .......... .......... .......... 1% 110M 5s 2024-03-04T18:07:20.270 INFO:teuthology.orchestra.run.smithi047.stderr: 1600K .......... .......... .......... .......... .......... 1% 107M 5s 2024-03-04T18:07:20.270 INFO:teuthology.orchestra.run.smithi047.stderr: 1650K .......... .......... .......... .......... .......... 1% 31.3M 5s 2024-03-04T18:07:20.270 INFO:teuthology.orchestra.run.smithi047.stderr: 1700K .......... .......... .......... .......... .......... 1% 67.3M 5s 2024-03-04T18:07:20.270 INFO:teuthology.orchestra.run.smithi047.stderr: 1750K .......... .......... .......... .......... .......... 1% 76.4M 5s 2024-03-04T18:07:20.270 INFO:teuthology.orchestra.run.smithi047.stderr: 1800K .......... .......... .......... .......... .......... 1% 56.3M 5s 2024-03-04T18:07:20.270 INFO:teuthology.orchestra.run.smithi047.stderr: 1850K .......... .......... .......... .......... .......... 2% 108M 5s 2024-03-04T18:07:20.270 INFO:teuthology.orchestra.run.smithi047.stderr: 1900K .......... .......... .......... .......... .......... 2% 27.2M 4s 2024-03-04T18:07:20.270 INFO:teuthology.orchestra.run.smithi047.stderr: 1950K .......... .......... .......... .......... .......... 2% 64.7M 4s 2024-03-04T18:07:20.275 INFO:teuthology.orchestra.run.smithi047.stderr: 2000K .......... .......... .......... .......... .......... 2% 10.4M 4s 2024-03-04T18:07:20.275 INFO:teuthology.orchestra.run.smithi047.stderr: 2050K .......... .......... .......... .......... .......... 2% 58.2M 4s 2024-03-04T18:07:20.275 INFO:teuthology.orchestra.run.smithi047.stderr: 2100K .......... .......... .......... .......... .......... 2% 98.4M 4s 2024-03-04T18:07:20.277 INFO:teuthology.orchestra.run.smithi047.stderr: 2150K .......... .......... .......... .......... .......... 2% 37.8M 4s 2024-03-04T18:07:20.278 INFO:teuthology.orchestra.run.smithi047.stderr: 2200K .......... .......... .......... .......... .......... 2% 28.6M 4s 2024-03-04T18:07:20.282 INFO:teuthology.orchestra.run.smithi047.stderr: 2250K .......... .......... .......... .......... .......... 2% 34.6M 4s 2024-03-04T18:07:20.282 INFO:teuthology.orchestra.run.smithi047.stderr: 2300K .......... .......... .......... .......... .......... 2% 106M 4s 2024-03-04T18:07:20.282 INFO:teuthology.orchestra.run.smithi047.stderr: 2350K .......... .......... .......... .......... .......... 2% 119M 4s 2024-03-04T18:07:20.282 INFO:teuthology.orchestra.run.smithi047.stderr: 2400K .......... .......... .......... .......... .......... 2% 65.5M 4s 2024-03-04T18:07:20.283 INFO:teuthology.orchestra.run.smithi047.stderr: 2450K .......... .......... .......... .......... .......... 2% 54.7M 4s 2024-03-04T18:07:20.286 INFO:teuthology.orchestra.run.smithi047.stderr: 2500K .......... .......... .......... .......... .......... 2% 41.5M 4s 2024-03-04T18:07:20.286 INFO:teuthology.orchestra.run.smithi047.stderr: 2550K .......... .......... .......... .......... .......... 2% 123M 4s 2024-03-04T18:07:20.286 INFO:teuthology.orchestra.run.smithi047.stderr: 2600K .......... .......... .......... .......... .......... 2% 114M 4s 2024-03-04T18:07:20.287 INFO:teuthology.orchestra.run.smithi047.stderr: 2650K .......... .......... .......... .......... .......... 2% 100M 4s 2024-03-04T18:07:20.287 INFO:teuthology.orchestra.run.smithi047.stderr: 2700K .......... .......... .......... .......... .......... 2% 33.8M 4s 2024-03-04T18:07:20.304 INFO:teuthology.orchestra.run.smithi047.stderr: 2750K .......... .......... .......... .......... .......... 3% 38.6M 4s 2024-03-04T18:07:20.304 INFO:teuthology.orchestra.run.smithi047.stderr: 2800K .......... .......... .......... .......... .......... 3% 120M 4s 2024-03-04T18:07:20.304 INFO:teuthology.orchestra.run.smithi047.stderr: 2850K .......... .......... .......... .......... .......... 3% 114M 4s 2024-03-04T18:07:20.304 INFO:teuthology.orchestra.run.smithi047.stderr: 2900K .......... .......... .......... .......... .......... 3% 86.6M 4s 2024-03-04T18:07:20.304 INFO:teuthology.orchestra.run.smithi047.stderr: 2950K .......... .......... .......... .......... .......... 3% 117M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3000K .......... .......... .......... .......... .......... 3% 47.4M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3050K .......... .......... .......... .......... .......... 3% 119M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3100K .......... .......... .......... .......... .......... 3% 94.2M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3150K .......... .......... .......... .......... .......... 3% 112M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3200K .......... .......... .......... .......... .......... 3% 65.0M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3250K .......... .......... .......... .......... .......... 3% 115M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3300K .......... .......... .......... .......... .......... 3% 83.1M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3350K .......... .......... .......... .......... .......... 3% 50.9M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3400K .......... .......... .......... .......... .......... 3% 46.6M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3450K .......... .......... .......... .......... .......... 3% 119M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3500K .......... .......... .......... .......... .......... 3% 109M 3s 2024-03-04T18:07:20.305 INFO:teuthology.orchestra.run.smithi047.stderr: 3550K .......... .......... .......... .......... .......... 3% 69.4M 3s 2024-03-04T18:07:20.306 INFO:teuthology.orchestra.run.smithi047.stderr: 3600K .......... .......... .......... .......... .......... 3% 97.0M 3s 2024-03-04T18:07:20.306 INFO:teuthology.orchestra.run.smithi047.stderr: 3650K .......... .......... .......... .......... .......... 3% 58.6M 3s 2024-03-04T18:07:20.306 INFO:teuthology.orchestra.run.smithi047.stderr: 3700K .......... .......... .......... .......... .......... 4% 54.8M 3s 2024-03-04T18:07:20.306 INFO:teuthology.orchestra.run.smithi047.stderr: 3750K .......... .......... .......... .......... .......... 4% 106M 3s 2024-03-04T18:07:20.306 INFO:teuthology.orchestra.run.smithi047.stderr: 3800K .......... .......... .......... .......... .......... 4% 38.6M 3s 2024-03-04T18:07:20.306 INFO:teuthology.orchestra.run.smithi047.stderr: 3850K .......... .......... .......... .......... .......... 4% 77.4M 3s 2024-03-04T18:07:20.309 INFO:teuthology.orchestra.run.smithi047.stderr: 3900K .......... .......... .......... .......... .......... 4% 9.76M 3s 2024-03-04T18:07:20.309 INFO:teuthology.orchestra.run.smithi047.stderr: 3950K .......... .......... .......... .......... .......... 4% 115M 3s 2024-03-04T18:07:20.309 INFO:teuthology.orchestra.run.smithi047.stderr: 4000K .......... .......... .......... .......... .......... 4% 113M 3s 2024-03-04T18:07:20.309 INFO:teuthology.orchestra.run.smithi047.stderr: 4050K .......... .......... .......... .......... .......... 4% 37.9M 3s 2024-03-04T18:07:20.311 INFO:teuthology.orchestra.run.smithi047.stderr: 4100K .......... .......... .......... .......... .......... 4% 30.1M 3s 2024-03-04T18:07:20.311 INFO:teuthology.orchestra.run.smithi047.stderr: 4150K .......... .......... .......... .......... .......... 4% 42.9M 3s 2024-03-04T18:07:20.312 INFO:teuthology.orchestra.run.smithi047.stderr: 4200K .......... .......... .......... .......... .......... 4% 95.3M 3s 2024-03-04T18:07:20.314 INFO:teuthology.orchestra.run.smithi047.stderr: 4250K .......... .......... .......... .......... .......... 4% 59.3M 3s 2024-03-04T18:07:20.314 INFO:teuthology.orchestra.run.smithi047.stderr: 4300K .......... .......... .......... .......... .......... 4% 98.3M 3s 2024-03-04T18:07:20.316 INFO:teuthology.orchestra.run.smithi047.stderr: 4350K .......... .......... .......... .......... .......... 4% 28.3M 3s 2024-03-04T18:07:20.316 INFO:teuthology.orchestra.run.smithi047.stderr: 4400K .......... .......... .......... .......... .......... 4% 67.4M 3s 2024-03-04T18:07:20.316 INFO:teuthology.orchestra.run.smithi047.stderr: 4450K .......... .......... .......... .......... .......... 4% 56.5M 3s 2024-03-04T18:07:20.324 INFO:teuthology.orchestra.run.smithi047.stderr: 4500K .......... .......... .......... .......... .......... 4% 44.3M 3s 2024-03-04T18:07:20.325 INFO:teuthology.orchestra.run.smithi047.stderr: 4550K .......... .......... .......... .......... .......... 4% 107M 3s 2024-03-04T18:07:20.325 INFO:teuthology.orchestra.run.smithi047.stderr: 4600K .......... .......... .......... .......... .......... 5% 106M 3s 2024-03-04T18:07:20.325 INFO:teuthology.orchestra.run.smithi047.stderr: 4650K .......... .......... .......... .......... .......... 5% 116M 3s 2024-03-04T18:07:20.325 INFO:teuthology.orchestra.run.smithi047.stderr: 4700K .......... .......... .......... .......... .......... 5% 25.1M 3s 2024-03-04T18:07:20.325 INFO:teuthology.orchestra.run.smithi047.stderr: 4750K .......... .......... .......... .......... .......... 5% 98.3M 3s 2024-03-04T18:07:20.325 INFO:teuthology.orchestra.run.smithi047.stderr: 4800K .......... .......... .......... .......... .......... 5% 70.6M 3s 2024-03-04T18:07:20.325 INFO:teuthology.orchestra.run.smithi047.stderr: 4850K .......... .......... .......... .......... .......... 5% 122M 3s 2024-03-04T18:07:20.325 INFO:teuthology.orchestra.run.smithi047.stderr: 4900K .......... .......... .......... .......... .......... 5% 44.6M 3s 2024-03-04T18:07:20.325 INFO:teuthology.orchestra.run.smithi047.stderr: 4950K .......... .......... .......... .......... .......... 5% 106M 3s 2024-03-04T18:07:20.338 INFO:teuthology.orchestra.run.smithi047.stderr: 5000K .......... .......... .......... .......... .......... 5% 30.0M 3s 2024-03-04T18:07:20.338 INFO:teuthology.orchestra.run.smithi047.stderr: 5050K .......... .......... .......... .......... .......... 5% 104M 3s 2024-03-04T18:07:20.338 INFO:teuthology.orchestra.run.smithi047.stderr: 5100K .......... .......... .......... .......... .......... 5% 112M 3s 2024-03-04T18:07:20.338 INFO:teuthology.orchestra.run.smithi047.stderr: 5150K .......... .......... .......... .......... .......... 5% 46.1M 3s 2024-03-04T18:07:20.338 INFO:teuthology.orchestra.run.smithi047.stderr: 5200K .......... .......... .......... .......... .......... 5% 81.6M 3s 2024-03-04T18:07:20.338 INFO:teuthology.orchestra.run.smithi047.stderr: 5250K .......... .......... .......... .......... .......... 5% 54.4M 3s 2024-03-04T18:07:20.338 INFO:teuthology.orchestra.run.smithi047.stderr: 5300K .......... .......... .......... .......... .......... 5% 102M 3s 2024-03-04T18:07:20.338 INFO:teuthology.orchestra.run.smithi047.stderr: 5350K .......... .......... .......... .......... .......... 5% 105M 3s 2024-03-04T18:07:20.338 INFO:teuthology.orchestra.run.smithi047.stderr: 5400K .......... .......... .......... .......... .......... 5% 36.2M 3s 2024-03-04T18:07:20.339 INFO:teuthology.orchestra.run.smithi047.stderr: 5450K .......... .......... .......... .......... .......... 5% 117M 3s 2024-03-04T18:07:20.339 INFO:teuthology.orchestra.run.smithi047.stderr: 5500K .......... .......... .......... .......... .......... 5% 41.0M 3s 2024-03-04T18:07:20.339 INFO:teuthology.orchestra.run.smithi047.stderr: 5550K .......... .......... .......... .......... .......... 6% 112M 3s 2024-03-04T18:07:20.339 INFO:teuthology.orchestra.run.smithi047.stderr: 5600K .......... .......... .......... .......... .......... 6% 75.6M 2s 2024-03-04T18:07:20.339 INFO:teuthology.orchestra.run.smithi047.stderr: 5650K .......... .......... .......... .......... .......... 6% 68.1M 2s 2024-03-04T18:07:20.339 INFO:teuthology.orchestra.run.smithi047.stderr: 5700K .......... .......... .......... .......... .......... 6% 55.2M 2s 2024-03-04T18:07:20.339 INFO:teuthology.orchestra.run.smithi047.stderr: 5750K .......... .......... .......... .......... .......... 6% 43.8M 2s 2024-03-04T18:07:20.339 INFO:teuthology.orchestra.run.smithi047.stderr: 5800K .......... .......... .......... .......... .......... 6% 99.8M 2s 2024-03-04T18:07:20.341 INFO:teuthology.orchestra.run.smithi047.stderr: 5850K .......... .......... .......... .......... .......... 6% 23.4M 2s 2024-03-04T18:07:20.341 INFO:teuthology.orchestra.run.smithi047.stderr: 5900K .......... .......... .......... .......... .......... 6% 71.4M 2s 2024-03-04T18:07:20.345 INFO:teuthology.orchestra.run.smithi047.stderr: 5950K .......... .......... .......... .......... .......... 6% 16.1M 2s 2024-03-04T18:07:20.345 INFO:teuthology.orchestra.run.smithi047.stderr: 6000K .......... .......... .......... .......... .......... 6% 53.8M 2s 2024-03-04T18:07:20.346 INFO:teuthology.orchestra.run.smithi047.stderr: 6050K .......... .......... .......... .......... .......... 6% 93.8M 2s 2024-03-04T18:07:20.346 INFO:teuthology.orchestra.run.smithi047.stderr: 6100K .......... .......... .......... .......... .......... 6% 109M 2s 2024-03-04T18:07:20.346 INFO:teuthology.orchestra.run.smithi047.stderr: 6150K .......... .......... .......... .......... .......... 6% 26.2M 2s 2024-03-04T18:07:20.349 INFO:teuthology.orchestra.run.smithi047.stderr: 6200K .......... .......... .......... .......... .......... 6% 33.7M 2s 2024-03-04T18:07:20.349 INFO:teuthology.orchestra.run.smithi047.stderr: 6250K .......... .......... .......... .......... .......... 6% 58.6M 2s 2024-03-04T18:07:20.350 INFO:teuthology.orchestra.run.smithi047.stderr: 6300K .......... .......... .......... .......... .......... 6% 93.0M 2s 2024-03-04T18:07:20.352 INFO:teuthology.orchestra.run.smithi047.stderr: 6350K .......... .......... .......... .......... .......... 6% 33.1M 2s 2024-03-04T18:07:20.352 INFO:teuthology.orchestra.run.smithi047.stderr: 6400K .......... .......... .......... .......... .......... 6% 53.3M 2s 2024-03-04T18:07:20.354 INFO:teuthology.orchestra.run.smithi047.stderr: 6450K .......... .......... .......... .......... .......... 6% 26.9M 2s 2024-03-04T18:07:20.376 INFO:teuthology.orchestra.run.smithi047.stderr: 6500K .......... .......... .......... .......... .......... 7% 42.4M 2s 2024-03-04T18:07:20.376 INFO:teuthology.orchestra.run.smithi047.stderr: 6550K .......... .......... .......... .......... .......... 7% 105M 2s 2024-03-04T18:07:20.376 INFO:teuthology.orchestra.run.smithi047.stderr: 6600K .......... .......... .......... .......... .......... 7% 112M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 6650K .......... .......... .......... .......... .......... 7% 99.8M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 6700K .......... .......... .......... .......... .......... 7% 50.3M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 6750K .......... .......... .......... .......... .......... 7% 76.2M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 6800K .......... .......... .......... .......... .......... 7% 45.6M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 6850K .......... .......... .......... .......... .......... 7% 47.2M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 6900K .......... .......... .......... .......... .......... 7% 121M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 6950K .......... .......... .......... .......... .......... 7% 50.9M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 7000K .......... .......... .......... .......... .......... 7% 100M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 7050K .......... .......... .......... .......... .......... 7% 125M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 7100K .......... .......... .......... .......... .......... 7% 43.2M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 7150K .......... .......... .......... .......... .......... 7% 110M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 7200K .......... .......... .......... .......... .......... 7% 34.1M 2s 2024-03-04T18:07:20.377 INFO:teuthology.orchestra.run.smithi047.stderr: 7250K .......... .......... .......... .......... .......... 7% 111M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7300K .......... .......... .......... .......... .......... 7% 98.9M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7350K .......... .......... .......... .......... .......... 7% 39.7M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7400K .......... .......... .......... .......... .......... 8% 80.9M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7450K .......... .......... .......... .......... .......... 8% 85.2M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7500K .......... .......... .......... .......... .......... 8% 79.7M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7550K .......... .......... .......... .......... .......... 8% 71.5M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7600K .......... .......... .......... .......... .......... 8% 39.6M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7650K .......... .......... .......... .......... .......... 8% 83.6M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7700K .......... .......... .......... .......... .......... 8% 97.7M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7750K .......... .......... .......... .......... .......... 8% 36.4M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7800K .......... .......... .......... .......... .......... 8% 48.8M 2s 2024-03-04T18:07:20.378 INFO:teuthology.orchestra.run.smithi047.stderr: 7850K .......... .......... .......... .......... .......... 8% 47.8M 2s 2024-03-04T18:07:20.385 INFO:teuthology.orchestra.run.smithi047.stderr: 7900K .......... .......... .......... .......... .......... 8% 29.7M 2s 2024-03-04T18:07:20.385 INFO:teuthology.orchestra.run.smithi047.stderr: 7950K .......... .......... .......... .......... .......... 8% 46.0M 2s 2024-03-04T18:07:20.385 INFO:teuthology.orchestra.run.smithi047.stderr: 8000K .......... .......... .......... .......... .......... 8% 111M 2s 2024-03-04T18:07:20.385 INFO:teuthology.orchestra.run.smithi047.stderr: 8050K .......... .......... .......... .......... .......... 8% 26.0M 2s 2024-03-04T18:07:20.385 INFO:teuthology.orchestra.run.smithi047.stderr: 8100K .......... .......... .......... .......... .......... 8% 33.9M 2s 2024-03-04T18:07:20.385 INFO:teuthology.orchestra.run.smithi047.stderr: 8150K .......... .......... .......... .......... .......... 8% 64.3M 2s 2024-03-04T18:07:20.385 INFO:teuthology.orchestra.run.smithi047.stderr: 8200K .......... .......... .......... .......... .......... 8% 89.9M 2s 2024-03-04T18:07:20.385 INFO:teuthology.orchestra.run.smithi047.stderr: 8250K .......... .......... .......... .......... .......... 8% 61.1M 2s 2024-03-04T18:07:20.391 INFO:teuthology.orchestra.run.smithi047.stderr: 8300K .......... .......... .......... .......... .......... 8% 27.4M 2s 2024-03-04T18:07:20.391 INFO:teuthology.orchestra.run.smithi047.stderr: 8350K .......... .......... .......... .......... .......... 9% 95.5M 2s 2024-03-04T18:07:20.391 INFO:teuthology.orchestra.run.smithi047.stderr: 8400K .......... .......... .......... .......... .......... 9% 124M 2s 2024-03-04T18:07:20.391 INFO:teuthology.orchestra.run.smithi047.stderr: 8450K .......... .......... .......... .......... .......... 9% 45.6M 2s 2024-03-04T18:07:20.391 INFO:teuthology.orchestra.run.smithi047.stderr: 8500K .......... .......... .......... .......... .......... 9% 34.5M 2s 2024-03-04T18:07:20.391 INFO:teuthology.orchestra.run.smithi047.stderr: 8550K .......... .......... .......... .......... .......... 9% 73.0M 2s 2024-03-04T18:07:20.391 INFO:teuthology.orchestra.run.smithi047.stderr: 8600K .......... .......... .......... .......... .......... 9% 115M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 8650K .......... .......... .......... .......... .......... 9% 30.9M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 8700K .......... .......... .......... .......... .......... 9% 32.2M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 8750K .......... .......... .......... .......... .......... 9% 85.2M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 8800K .......... .......... .......... .......... .......... 9% 54.9M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 8850K .......... .......... .......... .......... .......... 9% 116M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 8900K .......... .......... .......... .......... .......... 9% 46.4M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 8950K .......... .......... .......... .......... .......... 9% 67.5M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 9000K .......... .......... .......... .......... .......... 9% 56.5M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 9050K .......... .......... .......... .......... .......... 9% 107M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 9100K .......... .......... .......... .......... .......... 9% 89.0M 2s 2024-03-04T18:07:20.405 INFO:teuthology.orchestra.run.smithi047.stderr: 9150K .......... .......... .......... .......... .......... 9% 41.3M 2s 2024-03-04T18:07:20.406 INFO:teuthology.orchestra.run.smithi047.stderr: 9200K .......... .......... .......... .......... .......... 9% 101M 2s 2024-03-04T18:07:20.406 INFO:teuthology.orchestra.run.smithi047.stderr: 9250K .......... .......... .......... .......... .......... 10% 59.0M 2s 2024-03-04T18:07:20.406 INFO:teuthology.orchestra.run.smithi047.stderr: 9300K .......... .......... .......... .......... .......... 10% 106M 2s 2024-03-04T18:07:20.406 INFO:teuthology.orchestra.run.smithi047.stderr: 9350K .......... .......... .......... .......... .......... 10% 50.0M 2s 2024-03-04T18:07:20.406 INFO:teuthology.orchestra.run.smithi047.stderr: 9400K .......... .......... .......... .......... .......... 10% 59.5M 2s 2024-03-04T18:07:20.406 INFO:teuthology.orchestra.run.smithi047.stderr: 9450K .......... .......... .......... .......... .......... 10% 62.7M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9500K .......... .......... .......... .......... .......... 10% 39.3M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9550K .......... .......... .......... .......... .......... 10% 111M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9600K .......... .......... .......... .......... .......... 10% 88.3M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9650K .......... .......... .......... .......... .......... 10% 93.7M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9700K .......... .......... .......... .......... .......... 10% 31.5M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9750K .......... .......... .......... .......... .......... 10% 42.8M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9800K .......... .......... .......... .......... .......... 10% 99.1M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9850K .......... .......... .......... .......... .......... 10% 96.4M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9900K .......... .......... .......... .......... .......... 10% 49.4M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 9950K .......... .......... .......... .......... .......... 10% 31.5M 2s 2024-03-04T18:07:20.416 INFO:teuthology.orchestra.run.smithi047.stderr: 10000K .......... .......... .......... .......... .......... 10% 49.8M 2s 2024-03-04T18:07:20.417 INFO:teuthology.orchestra.run.smithi047.stderr: 10050K .......... .......... .......... .......... .......... 10% 122M 2s 2024-03-04T18:07:20.417 INFO:teuthology.orchestra.run.smithi047.stderr: 10100K .......... .......... .......... .......... .......... 10% 72.3M 2s 2024-03-04T18:07:20.418 INFO:teuthology.orchestra.run.smithi047.stderr: 10150K .......... .......... .......... .......... .......... 10% 37.1M 2s 2024-03-04T18:07:20.419 INFO:teuthology.orchestra.run.smithi047.stderr: 10200K .......... .......... .......... .......... .......... 11% 38.2M 2s 2024-03-04T18:07:20.419 INFO:teuthology.orchestra.run.smithi047.stderr: 10250K .......... .......... .......... .......... .......... 11% 106M 2s 2024-03-04T18:07:20.420 INFO:teuthology.orchestra.run.smithi047.stderr: 10300K .......... .......... .......... .......... .......... 11% 29.0M 2s 2024-03-04T18:07:20.422 INFO:teuthology.orchestra.run.smithi047.stderr: 10350K .......... .......... .......... .......... .......... 11% 29.5M 2s 2024-03-04T18:07:20.423 INFO:teuthology.orchestra.run.smithi047.stderr: 10400K .......... .......... .......... .......... .......... 11% 45.7M 2s 2024-03-04T18:07:20.425 INFO:teuthology.orchestra.run.smithi047.stderr: 10450K .......... .......... .......... .......... .......... 11% 39.0M 2s 2024-03-04T18:07:20.425 INFO:teuthology.orchestra.run.smithi047.stderr: 10500K .......... .......... .......... .......... .......... 11% 101M 2s 2024-03-04T18:07:20.427 INFO:teuthology.orchestra.run.smithi047.stderr: 10550K .......... .......... .......... .......... .......... 11% 42.4M 2s 2024-03-04T18:07:20.427 INFO:teuthology.orchestra.run.smithi047.stderr: 10600K .......... .......... .......... .......... .......... 11% 74.1M 2s 2024-03-04T18:07:20.428 INFO:teuthology.orchestra.run.smithi047.stderr: 10650K .......... .......... .......... .......... .......... 11% 36.5M 2s 2024-03-04T18:07:20.432 INFO:teuthology.orchestra.run.smithi047.stderr: 10700K .......... .......... .......... .......... .......... 11% 25.3M 2s 2024-03-04T18:07:20.432 INFO:teuthology.orchestra.run.smithi047.stderr: 10750K .......... .......... .......... .......... .......... 11% 108M 2s 2024-03-04T18:07:20.432 INFO:teuthology.orchestra.run.smithi047.stderr: 10800K .......... .......... .......... .......... .......... 11% 97.8M 2s 2024-03-04T18:07:20.433 INFO:teuthology.orchestra.run.smithi047.stderr: 10850K .......... .......... .......... .......... .......... 11% 34.5M 2s 2024-03-04T18:07:20.436 INFO:teuthology.orchestra.run.smithi047.stderr: 10900K .......... .......... .......... .......... .......... 11% 39.6M 2s 2024-03-04T18:07:20.437 INFO:teuthology.orchestra.run.smithi047.stderr: 10950K .......... .......... .......... .......... .......... 11% 91.6M 2s 2024-03-04T18:07:20.437 INFO:teuthology.orchestra.run.smithi047.stderr: 11000K .......... .......... .......... .......... .......... 11% 68.5M 2s 2024-03-04T18:07:20.437 INFO:teuthology.orchestra.run.smithi047.stderr: 11050K .......... .......... .......... .......... .......... 11% 78.8M 2s 2024-03-04T18:07:20.437 INFO:teuthology.orchestra.run.smithi047.stderr: 11100K .......... .......... .......... .......... .......... 12% 48.8M 2s 2024-03-04T18:07:20.454 INFO:teuthology.orchestra.run.smithi047.stderr: 11150K .......... .......... .......... .......... .......... 12% 43.9M 2s 2024-03-04T18:07:20.454 INFO:teuthology.orchestra.run.smithi047.stderr: 11200K .......... .......... .......... .......... .......... 12% 107M 2s 2024-03-04T18:07:20.454 INFO:teuthology.orchestra.run.smithi047.stderr: 11250K .......... .......... .......... .......... .......... 12% 119M 2s 2024-03-04T18:07:20.454 INFO:teuthology.orchestra.run.smithi047.stderr: 11300K .......... .......... .......... .......... .......... 12% 42.8M 2s 2024-03-04T18:07:20.454 INFO:teuthology.orchestra.run.smithi047.stderr: 11350K .......... .......... .......... .......... .......... 12% 92.9M 2s 2024-03-04T18:07:20.454 INFO:teuthology.orchestra.run.smithi047.stderr: 11400K .......... .......... .......... .......... .......... 12% 112M 2s 2024-03-04T18:07:20.454 INFO:teuthology.orchestra.run.smithi047.stderr: 11450K .......... .......... .......... .......... .......... 12% 58.1M 2s 2024-03-04T18:07:20.454 INFO:teuthology.orchestra.run.smithi047.stderr: 11500K .......... .......... .......... .......... .......... 12% 60.9M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 11550K .......... .......... .......... .......... .......... 12% 50.0M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 11600K .......... .......... .......... .......... .......... 12% 48.2M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 11650K .......... .......... .......... .......... .......... 12% 112M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 11700K .......... .......... .......... .......... .......... 12% 59.5M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 11750K .......... .......... .......... .......... .......... 12% 54.3M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 11800K .......... .......... .......... .......... .......... 12% 49.9M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 11850K .......... .......... .......... .......... .......... 12% 81.7M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 11900K .......... .......... .......... .......... .......... 12% 121M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 11950K .......... .......... .......... .......... .......... 12% 59.0M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 12000K .......... .......... .......... .......... .......... 12% 36.9M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 12050K .......... .......... .......... .......... .......... 13% 34.2M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 12100K .......... .......... .......... .......... .......... 13% 115M 2s 2024-03-04T18:07:20.455 INFO:teuthology.orchestra.run.smithi047.stderr: 12150K .......... .......... .......... .......... .......... 13% 83.0M 2s 2024-03-04T18:07:20.456 INFO:teuthology.orchestra.run.smithi047.stderr: 12200K .......... .......... .......... .......... .......... 13% 63.8M 2s 2024-03-04T18:07:20.465 INFO:teuthology.orchestra.run.smithi047.stderr: 12250K .......... .......... .......... .......... .......... 13% 27.6M 2s 2024-03-04T18:07:20.465 INFO:teuthology.orchestra.run.smithi047.stderr: 12300K .......... .......... .......... .......... .......... 13% 49.9M 2s 2024-03-04T18:07:20.465 INFO:teuthology.orchestra.run.smithi047.stderr: 12350K .......... .......... .......... .......... .......... 13% 109M 2s 2024-03-04T18:07:20.465 INFO:teuthology.orchestra.run.smithi047.stderr: 12400K .......... .......... .......... .......... .......... 13% 86.0M 2s 2024-03-04T18:07:20.465 INFO:teuthology.orchestra.run.smithi047.stderr: 12450K .......... .......... .......... .......... .......... 13% 45.3M 2s 2024-03-04T18:07:20.465 INFO:teuthology.orchestra.run.smithi047.stderr: 12500K .......... .......... .......... .......... .......... 13% 53.8M 2s 2024-03-04T18:07:20.465 INFO:teuthology.orchestra.run.smithi047.stderr: 12550K .......... .......... .......... .......... .......... 13% 124M 2s 2024-03-04T18:07:20.466 INFO:teuthology.orchestra.run.smithi047.stderr: 12600K .......... .......... .......... .......... .......... 13% 38.3M 2s 2024-03-04T18:07:20.466 INFO:teuthology.orchestra.run.smithi047.stderr: 12650K .......... .......... .......... .......... .......... 13% 33.8M 2s 2024-03-04T18:07:20.466 INFO:teuthology.orchestra.run.smithi047.stderr: 12700K .......... .......... .......... .......... .......... 13% 35.0M 2s 2024-03-04T18:07:20.466 INFO:teuthology.orchestra.run.smithi047.stderr: 12750K .......... .......... .......... .......... .......... 13% 92.7M 2s 2024-03-04T18:07:20.466 INFO:teuthology.orchestra.run.smithi047.stderr: 12800K .......... .......... .......... .......... .......... 13% 98.8M 2s 2024-03-04T18:07:20.469 INFO:teuthology.orchestra.run.smithi047.stderr: 12850K .......... .......... .......... .......... .......... 13% 37.3M 2s 2024-03-04T18:07:20.470 INFO:teuthology.orchestra.run.smithi047.stderr: 12900K .......... .......... .......... .......... .......... 13% 39.2M 2s 2024-03-04T18:07:20.470 INFO:teuthology.orchestra.run.smithi047.stderr: 12950K .......... .......... .......... .......... .......... 13% 54.6M 2s 2024-03-04T18:07:20.470 INFO:teuthology.orchestra.run.smithi047.stderr: 13000K .......... .......... .......... .......... .......... 14% 44.3M 2s 2024-03-04T18:07:20.471 INFO:teuthology.orchestra.run.smithi047.stderr: 13050K .......... .......... .......... .......... .......... 14% 30.4M 2s 2024-03-04T18:07:20.471 INFO:teuthology.orchestra.run.smithi047.stderr: 13100K .......... .......... .......... .......... .......... 14% 110M 2s 2024-03-04T18:07:20.473 INFO:teuthology.orchestra.run.smithi047.stderr: 13150K .......... .......... .......... .......... .......... 14% 41.0M 2s 2024-03-04T18:07:20.474 INFO:teuthology.orchestra.run.smithi047.stderr: 13200K .......... .......... .......... .......... .......... 14% 41.4M 2s 2024-03-04T18:07:21.736 INFO:teuthology.orchestra.run.smithi047.stderr: 13250K .......... .......... .......... .......... .......... 14% 41.6M 2s 2024-03-04T18:07:21.736 INFO:teuthology.orchestra.run.smithi047.stderr: 13300K .......... .......... .......... .......... .......... 14% 113M 2s 2024-03-04T18:07:21.736 INFO:teuthology.orchestra.run.smithi047.stderr: 13350K .......... .......... .......... .......... .......... 14% 77.1M 2s 2024-03-04T18:07:21.736 INFO:teuthology.orchestra.run.smithi047.stderr: 13400K .......... .......... .......... .......... .......... 14% 43.0M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13450K .......... .......... .......... .......... .......... 14% 85.2M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13500K .......... .......... .......... .......... .......... 14% 60.8M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13550K .......... .......... .......... .......... .......... 14% 108M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13600K .......... .......... .......... .......... .......... 14% 56.1M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13650K .......... .......... .......... .......... .......... 14% 111M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13700K .......... .......... .......... .......... .......... 14% 55.5M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13750K .......... .......... .......... .......... .......... 14% 94.5M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13800K .......... .......... .......... .......... .......... 14% 38.9M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13850K .......... .......... .......... .......... .......... 14% 84.5M 2s 2024-03-04T18:07:21.737 INFO:teuthology.orchestra.run.smithi047.stderr: 13900K .......... .......... .......... .......... .......... 15% 40.8M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 13950K .......... .......... .......... .......... .......... 15% 78.6M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14000K .......... .......... .......... .......... .......... 15% 104M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14050K .......... .......... .......... .......... .......... 15% 98.3M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14100K .......... .......... .......... .......... .......... 15% 61.8M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14150K .......... .......... .......... .......... .......... 15% 45.7M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14200K .......... .......... .......... .......... .......... 15% 64.7M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14250K .......... .......... .......... .......... .......... 15% 71.2M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14300K .......... .......... .......... .......... .......... 15% 43.1M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14350K .......... .......... .......... .......... .......... 15% 64.9M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14400K .......... .......... .......... .......... .......... 15% 48.2M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14450K .......... .......... .......... .......... .......... 15% 105M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14500K .......... .......... .......... .......... .......... 15% 48.5M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14550K .......... .......... .......... .......... .......... 15% 32.8M 2s 2024-03-04T18:07:21.738 INFO:teuthology.orchestra.run.smithi047.stderr: 14600K .......... .......... .......... .......... .......... 15% 63.4M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 14650K .......... .......... .......... .......... .......... 15% 47.7M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 14700K .......... .......... .......... .......... .......... 15% 80.9M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 14750K .......... .......... .......... .......... .......... 15% 40.8M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 14800K .......... .......... .......... .......... .......... 15% 71.4M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 14850K .......... .......... .......... .......... .......... 16% 81.4M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 14900K .......... .......... .......... .......... .......... 16% 98.0M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 14950K .......... .......... .......... .......... .......... 16% 31.8M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 15000K .......... .......... .......... .......... .......... 16% 66.4M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 15050K .......... .......... .......... .......... .......... 16% 81.5M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 15100K .......... .......... .......... .......... .......... 16% 38.7M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 15150K .......... .......... .......... .......... .......... 16% 83.8M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 15200K .......... .......... .......... .......... .......... 16% 30.2M 2s 2024-03-04T18:07:21.739 INFO:teuthology.orchestra.run.smithi047.stderr: 15250K .......... .......... .......... .......... .......... 16% 80.3M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15300K .......... .......... .......... .......... .......... 16% 40.4M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15350K .......... .......... .......... .......... .......... 16% 70.5M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15400K .......... .......... .......... .......... .......... 16% 78.4M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15450K .......... .......... .......... .......... .......... 16% 45.8M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15500K .......... .......... .......... .......... .......... 16% 50.6M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15550K .......... .......... .......... .......... .......... 16% 37.9M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15600K .......... .......... .......... .......... .......... 16% 59.0M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15650K .......... .......... .......... .......... .......... 16% 63.3M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15700K .......... .......... .......... .......... .......... 16% 42.7M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15750K .......... .......... .......... .......... .......... 17% 37.9M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15800K .......... .......... .......... .......... .......... 17% 80.9M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15850K .......... .......... .......... .......... .......... 17% 81.5M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15900K .......... .......... .......... .......... .......... 17% 84.2M 2s 2024-03-04T18:07:21.740 INFO:teuthology.orchestra.run.smithi047.stderr: 15950K .......... .......... .......... .......... .......... 17% 53.5M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16000K .......... .......... .......... .......... .......... 17% 83.3M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16050K .......... .......... .......... .......... .......... 17% 68.8M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16100K .......... .......... .......... .......... .......... 17% 75.5M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16150K .......... .......... .......... .......... .......... 17% 86.0M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16200K .......... .......... .......... .......... .......... 17% 77.4M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16250K .......... .......... .......... .......... .......... 17% 32.2M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16300K .......... .......... .......... .......... .......... 17% 73.7M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16350K .......... .......... .......... .......... .......... 17% 78.6M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16400K .......... .......... .......... .......... .......... 17% 77.9M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16450K .......... .......... .......... .......... .......... 17% 58.0M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16500K .......... .......... .......... .......... .......... 17% 60.6M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16550K .......... .......... .......... .......... .......... 17% 52.1M 2s 2024-03-04T18:07:21.741 INFO:teuthology.orchestra.run.smithi047.stderr: 16600K .......... .......... .......... .......... .......... 17% 87.8M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 16650K .......... .......... .......... .......... .......... 17% 42.8M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 16700K .......... .......... .......... .......... .......... 18% 48.9M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 16750K .......... .......... .......... .......... .......... 18% 65.7M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 16800K .......... .......... .......... .......... .......... 18% 86.2M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 16850K .......... .......... .......... .......... .......... 18% 36.9M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 16900K .......... .......... .......... .......... .......... 18% 39.0M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 16950K .......... .......... .......... .......... .......... 18% 82.7M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 17000K .......... .......... .......... .......... .......... 18% 41.2M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 17050K .......... .......... .......... .......... .......... 18% 87.5M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 17100K .......... .......... .......... .......... .......... 18% 34.2M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 17150K .......... .......... .......... .......... .......... 18% 80.4M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 17200K .......... .......... .......... .......... .......... 18% 40.5M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 17250K .......... .......... .......... .......... .......... 18% 42.3M 2s 2024-03-04T18:07:21.742 INFO:teuthology.orchestra.run.smithi047.stderr: 17300K .......... .......... .......... .......... .......... 18% 70.4M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17350K .......... .......... .......... .......... .......... 18% 99.3M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17400K .......... .......... .......... .......... .......... 18% 83.3M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17450K .......... .......... .......... .......... .......... 18% 61.4M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17500K .......... .......... .......... .......... .......... 18% 77.8M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17550K .......... .......... .......... .......... .......... 18% 40.3M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17600K .......... .......... .......... .......... .......... 18% 81.7M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17650K .......... .......... .......... .......... .......... 19% 41.5M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17700K .......... .......... .......... .......... .......... 19% 61.9M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17750K .......... .......... .......... .......... .......... 19% 97.6M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17800K .......... .......... .......... .......... .......... 19% 35.1M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17850K .......... .......... .......... .......... .......... 19% 73.5M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17900K .......... .......... .......... .......... .......... 19% 73.3M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 17950K .......... .......... .......... .......... .......... 19% 88.7M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 18000K .......... .......... .......... .......... .......... 19% 35.3M 2s 2024-03-04T18:07:21.743 INFO:teuthology.orchestra.run.smithi047.stderr: 18050K .......... .......... .......... .......... .......... 19% 84.8M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18100K .......... .......... .......... .......... .......... 19% 68.7M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18150K .......... .......... .......... .......... .......... 19% 47.2M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18200K .......... .......... .......... .......... .......... 19% 65.4M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18250K .......... .......... .......... .......... .......... 19% 81.5M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18300K .......... .......... .......... .......... .......... 19% 47.4M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18350K .......... .......... .......... .......... .......... 19% 95.4M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18400K .......... .......... .......... .......... .......... 19% 100M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18450K .......... .......... .......... .......... .......... 19% 44.3M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18500K .......... .......... .......... .......... .......... 19% 95.2M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18550K .......... .......... .......... .......... .......... 20% 80.4M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18600K .......... .......... .......... .......... .......... 20% 41.8M 2s 2024-03-04T18:07:21.744 INFO:teuthology.orchestra.run.smithi047.stderr: 18650K .......... .......... .......... .......... .......... 20% 64.7M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 18700K .......... .......... .......... .......... .......... 20% 80.8M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 18750K .......... .......... .......... .......... .......... 20% 77.5M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 18800K .......... .......... .......... .......... .......... 20% 76.8M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 18850K .......... .......... .......... .......... .......... 20% 29.3M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 18900K .......... .......... .......... .......... .......... 20% 77.2M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 18950K .......... .......... .......... .......... .......... 20% 55.8M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 19000K .......... .......... .......... .......... .......... 20% 72.4M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 19050K .......... .......... .......... .......... .......... 20% 57.0M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 19100K .......... .......... .......... .......... .......... 20% 93.7M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 19150K .......... .......... .......... .......... .......... 20% 62.3M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 19200K .......... .......... .......... .......... .......... 20% 61.4M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 19250K .......... .......... .......... .......... .......... 20% 62.3M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 19300K .......... .......... .......... .......... .......... 20% 31.3M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 19350K .......... .......... .......... .......... .......... 20% 104M 2s 2024-03-04T18:07:21.745 INFO:teuthology.orchestra.run.smithi047.stderr: 19400K .......... .......... .......... .......... .......... 20% 60.7M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19450K .......... .......... .......... .......... .......... 20% 77.1M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19500K .......... .......... .......... .......... .......... 21% 30.6M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19550K .......... .......... .......... .......... .......... 21% 44.4M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19600K .......... .......... .......... .......... .......... 21% 83.8M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19650K .......... .......... .......... .......... .......... 21% 78.6M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19700K .......... .......... .......... .......... .......... 21% 47.7M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19750K .......... .......... .......... .......... .......... 21% 44.9M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19800K .......... .......... .......... .......... .......... 21% 76.7M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19850K .......... .......... .......... .......... .......... 21% 105M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19900K .......... .......... .......... .......... .......... 21% 49.3M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 19950K .......... .......... .......... .......... .......... 21% 44.9M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 20000K .......... .......... .......... .......... .......... 21% 64.5M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 20050K .......... .......... .......... .......... .......... 21% 90.6M 2s 2024-03-04T18:07:21.746 INFO:teuthology.orchestra.run.smithi047.stderr: 20100K .......... .......... .......... .......... .......... 21% 100M 2s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20150K .......... .......... .......... .......... .......... 21% 61.0M 2s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20200K .......... .......... .......... .......... .......... 21% 45.0M 2s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20250K .......... .......... .......... .......... .......... 21% 58.9M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20300K .......... .......... .......... .......... .......... 21% 89.6M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20350K .......... .......... .......... .......... .......... 21% 59.7M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20400K .......... .......... .......... .......... .......... 22% 64.7M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20450K .......... .......... .......... .......... .......... 22% 80.2M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20500K .......... .......... .......... .......... .......... 22% 73.6M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20550K .......... .......... .......... .......... .......... 22% 52.8M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20600K .......... .......... .......... .......... .......... 22% 49.1M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20650K .......... .......... .......... .......... .......... 22% 58.4M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20700K .......... .......... .......... .......... .......... 22% 71.2M 1s 2024-03-04T18:07:21.747 INFO:teuthology.orchestra.run.smithi047.stderr: 20750K .......... .......... .......... .......... .......... 22% 91.5M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 20800K .......... .......... .......... .......... .......... 22% 67.3M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 20850K .......... .......... .......... .......... .......... 22% 49.2M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 20900K .......... .......... .......... .......... .......... 22% 46.6M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 20950K .......... .......... .......... .......... .......... 22% 74.4M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 21000K .......... .......... .......... .......... .......... 22% 85.3M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 21050K .......... .......... .......... .......... .......... 22% 72.3M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 21100K .......... .......... .......... .......... .......... 22% 42.1M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 21150K .......... .......... .......... .......... .......... 22% 66.7M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 21200K .......... .......... .......... .......... .......... 22% 71.8M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 21250K .......... .......... .......... .......... .......... 22% 111M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 21300K .......... .......... .......... .......... .......... 22% 65.4M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 21350K .......... .......... .......... .......... .......... 23% 40.9M 1s 2024-03-04T18:07:21.748 INFO:teuthology.orchestra.run.smithi047.stderr: 21400K .......... .......... .......... .......... .......... 23% 66.2M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21450K .......... .......... .......... .......... .......... 23% 70.9M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21500K .......... .......... .......... .......... .......... 23% 78.6M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21550K .......... .......... .......... .......... .......... 23% 78.4M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21600K .......... .......... .......... .......... .......... 23% 47.7M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21650K .......... .......... .......... .......... .......... 23% 69.8M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21700K .......... .......... .......... .......... .......... 23% 90.1M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21750K .......... .......... .......... .......... .......... 23% 25.3M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21800K .......... .......... .......... .......... .......... 23% 103M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21850K .......... .......... .......... .......... .......... 23% 79.0M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21900K .......... .......... .......... .......... .......... 23% 83.7M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 21950K .......... .......... .......... .......... .......... 23% 111M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 22000K .......... .......... .......... .......... .......... 23% 57.7M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 22050K .......... .......... .......... .......... .......... 23% 50.7M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 22100K .......... .......... .......... .......... .......... 23% 34.0M 1s 2024-03-04T18:07:21.749 INFO:teuthology.orchestra.run.smithi047.stderr: 22150K .......... .......... .......... .......... .......... 23% 93.9M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22200K .......... .......... .......... .......... .......... 23% 30.8M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22250K .......... .......... .......... .......... .......... 24% 102M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22300K .......... .......... .......... .......... .......... 24% 65.4M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22350K .......... .......... .......... .......... .......... 24% 45.7M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22400K .......... .......... .......... .......... .......... 24% 63.6M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22450K .......... .......... .......... .......... .......... 24% 102M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22500K .......... .......... .......... .......... .......... 24% 85.5M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22550K .......... .......... .......... .......... .......... 24% 41.7M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22600K .......... .......... .......... .......... .......... 24% 66.6M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22650K .......... .......... .......... .......... .......... 24% 94.0M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22700K .......... .......... .......... .......... .......... 24% 53.8M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22750K .......... .......... .......... .......... .......... 24% 50.7M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22800K .......... .......... .......... .......... .......... 24% 68.3M 1s 2024-03-04T18:07:21.750 INFO:teuthology.orchestra.run.smithi047.stderr: 22850K .......... .......... .......... .......... .......... 24% 131M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 22900K .......... .......... .......... .......... .......... 24% 64.7M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 22950K .......... .......... .......... .......... .......... 24% 61.1M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23000K .......... .......... .......... .......... .......... 24% 52.5M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23050K .......... .......... .......... .......... .......... 24% 66.5M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23100K .......... .......... .......... .......... .......... 24% 93.8M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23150K .......... .......... .......... .......... .......... 24% 59.2M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23200K .......... .......... .......... .......... .......... 25% 54.0M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23250K .......... .......... .......... .......... .......... 25% 60.7M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23300K .......... .......... .......... .......... .......... 25% 87.0M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23350K .......... .......... .......... .......... .......... 25% 67.8M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23400K .......... .......... .......... .......... .......... 25% 52.5M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23450K .......... .......... .......... .......... .......... 25% 57.0M 1s 2024-03-04T18:07:21.751 INFO:teuthology.orchestra.run.smithi047.stderr: 23500K .......... .......... .......... .......... .......... 25% 78.4M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 23550K .......... .......... .......... .......... .......... 25% 76.3M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 23600K .......... .......... .......... .......... .......... 25% 89.5M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 23650K .......... .......... .......... .......... .......... 25% 86.1M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 23700K .......... .......... .......... .......... .......... 25% 47.4M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 23750K .......... .......... .......... .......... .......... 25% 45.8M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 23800K .......... .......... .......... .......... .......... 25% 74.9M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 23850K .......... .......... .......... .......... .......... 25% 98.1M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 23900K .......... .......... .......... .......... .......... 25% 81.0M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 23950K .......... .......... .......... .......... .......... 25% 44.0M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 24000K .......... .......... .......... .......... .......... 25% 70.6M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 24050K .......... .......... .......... .......... .......... 25% 60.3M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 24100K .......... .......... .......... .......... .......... 25% 71.6M 1s 2024-03-04T18:07:21.752 INFO:teuthology.orchestra.run.smithi047.stderr: 24150K .......... .......... .......... .......... .......... 26% 51.9M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24200K .......... .......... .......... .......... .......... 26% 62.3M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24250K .......... .......... .......... .......... .......... 26% 58.9M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24300K .......... .......... .......... .......... .......... 26% 75.8M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24350K .......... .......... .......... .......... .......... 26% 70.0M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24400K .......... .......... .......... .......... .......... 26% 92.4M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24450K .......... .......... .......... .......... .......... 26% 78.9M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24500K .......... .......... .......... .......... .......... 26% 135M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24550K .......... .......... .......... .......... .......... 26% 37.4M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24600K .......... .......... .......... .......... .......... 26% 88.6M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24650K .......... .......... .......... .......... .......... 26% 38.2M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24700K .......... .......... .......... .......... .......... 26% 50.8M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24750K .......... .......... .......... .......... .......... 26% 96.7M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24800K .......... .......... .......... .......... .......... 26% 69.4M 1s 2024-03-04T18:07:21.753 INFO:teuthology.orchestra.run.smithi047.stderr: 24850K .......... .......... .......... .......... .......... 26% 79.6M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 24900K .......... .......... .......... .......... .......... 26% 51.3M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 24950K .......... .......... .......... .......... .......... 26% 57.8M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25000K .......... .......... .......... .......... .......... 26% 107M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25050K .......... .......... .......... .......... .......... 27% 38.9M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25100K .......... .......... .......... .......... .......... 27% 58.0M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25150K .......... .......... .......... .......... .......... 27% 76.1M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25200K .......... .......... .......... .......... .......... 27% 81.4M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25250K .......... .......... .......... .......... .......... 27% 39.5M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25300K .......... .......... .......... .......... .......... 27% 34.6M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25350K .......... .......... .......... .......... .......... 27% 69.8M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25400K .......... .......... .......... .......... .......... 27% 55.4M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25450K .......... .......... .......... .......... .......... 27% 96.8M 1s 2024-03-04T18:07:21.754 INFO:teuthology.orchestra.run.smithi047.stderr: 25500K .......... .......... .......... .......... .......... 27% 51.4M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 25550K .......... .......... .......... .......... .......... 27% 61.2M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 25600K .......... .......... .......... .......... .......... 27% 50.3M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 25650K .......... .......... .......... .......... .......... 27% 65.6M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 25700K .......... .......... .......... .......... .......... 27% 35.2M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 25750K .......... .......... .......... .......... .......... 27% 85.3M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 25800K .......... .......... .......... .......... .......... 27% 58.8M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 25850K .......... .......... .......... .......... .......... 27% 67.2M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 25900K .......... .......... .......... .......... .......... 27% 77.0M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 25950K .......... .......... .......... .......... .......... 27% 55.5M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 26000K .......... .......... .......... .......... .......... 28% 61.8M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 26050K .......... .......... .......... .......... .......... 28% 72.1M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 26100K .......... .......... .......... .......... .......... 28% 53.1M 1s 2024-03-04T18:07:21.755 INFO:teuthology.orchestra.run.smithi047.stderr: 26150K .......... .......... .......... .......... .......... 28% 72.6M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26200K .......... .......... .......... .......... .......... 28% 51.3M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26250K .......... .......... .......... .......... .......... 28% 54.9M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26300K .......... .......... .......... .......... .......... 28% 59.4M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26350K .......... .......... .......... .......... .......... 28% 55.6M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26400K .......... .......... .......... .......... .......... 28% 68.1M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26450K .......... .......... .......... .......... .......... 28% 62.5M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26500K .......... .......... .......... .......... .......... 28% 46.4M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26550K .......... .......... .......... .......... .......... 28% 77.6M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26600K .......... .......... .......... .......... .......... 28% 65.4M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26650K .......... .......... .......... .......... .......... 28% 55.3M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26700K .......... .......... .......... .......... .......... 28% 87.9M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26750K .......... .......... .......... .......... .......... 28% 47.2M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26800K .......... .......... .......... .......... .......... 28% 80.2M 1s 2024-03-04T18:07:21.756 INFO:teuthology.orchestra.run.smithi047.stderr: 26850K .......... .......... .......... .......... .......... 28% 56.6M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 26900K .......... .......... .......... .......... .......... 29% 70.9M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 26950K .......... .......... .......... .......... .......... 29% 105M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27000K .......... .......... .......... .......... .......... 29% 57.5M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27050K .......... .......... .......... .......... .......... 29% 66.3M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27100K .......... .......... .......... .......... .......... 29% 78.9M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27150K .......... .......... .......... .......... .......... 29% 65.8M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27200K .......... .......... .......... .......... .......... 29% 109M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27250K .......... .......... .......... .......... .......... 29% 66.0M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27300K .......... .......... .......... .......... .......... 29% 70.0M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27350K .......... .......... .......... .......... .......... 29% 64.0M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27400K .......... .......... .......... .......... .......... 29% 74.8M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27450K .......... .......... .......... .......... .......... 29% 63.6M 1s 2024-03-04T18:07:21.757 INFO:teuthology.orchestra.run.smithi047.stderr: 27500K .......... .......... .......... .......... .......... 29% 86.6M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 27550K .......... .......... .......... .......... .......... 29% 59.3M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 27600K .......... .......... .......... .......... .......... 29% 91.0M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 27650K .......... .......... .......... .......... .......... 29% 52.7M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 27700K .......... .......... .......... .......... .......... 29% 76.9M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 27750K .......... .......... .......... .......... .......... 29% 77.7M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 27800K .......... .......... .......... .......... .......... 29% 83.0M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 27850K .......... .......... .......... .......... .......... 30% 53.5M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 27900K .......... .......... .......... .......... .......... 30% 58.5M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 27950K .......... .......... .......... .......... .......... 30% 98.2M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 28000K .......... .......... .......... .......... .......... 30% 120M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 28050K .......... .......... .......... .......... .......... 30% 39.0M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 28100K .......... .......... .......... .......... .......... 30% 56.0M 1s 2024-03-04T18:07:21.758 INFO:teuthology.orchestra.run.smithi047.stderr: 28150K .......... .......... .......... .......... .......... 30% 58.0M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28200K .......... .......... .......... .......... .......... 30% 118M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28250K .......... .......... .......... .......... .......... 30% 38.1M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28300K .......... .......... .......... .......... .......... 30% 39.4M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28350K .......... .......... .......... .......... .......... 30% 44.3M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28400K .......... .......... .......... .......... .......... 30% 49.2M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28450K .......... .......... .......... .......... .......... 30% 48.1M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28500K .......... .......... .......... .......... .......... 30% 120M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28550K .......... .......... .......... .......... .......... 30% 89.8M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28600K .......... .......... .......... .......... .......... 30% 55.4M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28650K .......... .......... .......... .......... .......... 30% 45.2M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28700K .......... .......... .......... .......... .......... 30% 42.4M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28750K .......... .......... .......... .......... .......... 31% 112M 1s 2024-03-04T18:07:21.759 INFO:teuthology.orchestra.run.smithi047.stderr: 28800K .......... .......... .......... .......... .......... 31% 47.6M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 28850K .......... .......... .......... .......... .......... 31% 56.5M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 28900K .......... .......... .......... .......... .......... 31% 87.3M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 28950K .......... .......... .......... .......... .......... 31% 125M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 29000K .......... .......... .......... .......... .......... 31% 60.0M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 29050K .......... .......... .......... .......... .......... 31% 58.6M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 29100K .......... .......... .......... .......... .......... 31% 61.7M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 29150K .......... .......... .......... .......... .......... 31% 40.6M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 29200K .......... .......... .......... .......... .......... 31% 103M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 29250K .......... .......... .......... .......... .......... 31% 58.7M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 29300K .......... .......... .......... .......... .......... 31% 48.7M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 29350K .......... .......... .......... .......... .......... 31% 59.2M 1s 2024-03-04T18:07:21.760 INFO:teuthology.orchestra.run.smithi047.stderr: 29400K .......... .......... .......... .......... .......... 31% 130M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29450K .......... .......... .......... .......... .......... 31% 56.0M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29500K .......... .......... .......... .......... .......... 31% 40.7M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29550K .......... .......... .......... .......... .......... 31% 101M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29600K .......... .......... .......... .......... .......... 31% 57.0M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29650K .......... .......... .......... .......... .......... 31% 92.5M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29700K .......... .......... .......... .......... .......... 32% 74.1M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29750K .......... .......... .......... .......... .......... 32% 35.8M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29800K .......... .......... .......... .......... .......... 32% 99.9M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29850K .......... .......... .......... .......... .......... 32% 118M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29900K .......... .......... .......... .......... .......... 32% 77.2M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 29950K .......... .......... .......... .......... .......... 32% 43.3M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 30000K .......... .......... .......... .......... .......... 32% 81.7M 1s 2024-03-04T18:07:21.761 INFO:teuthology.orchestra.run.smithi047.stderr: 30050K .......... .......... .......... .......... .......... 32% 74.6M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30100K .......... .......... .......... .......... .......... 32% 108M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30150K .......... .......... .......... .......... .......... 32% 74.0M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30200K .......... .......... .......... .......... .......... 32% 35.8M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30250K .......... .......... .......... .......... .......... 32% 112M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30300K .......... .......... .......... .......... .......... 32% 82.2M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30350K .......... .......... .......... .......... .......... 32% 92.5M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30400K .......... .......... .......... .......... .......... 32% 61.1M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30450K .......... .......... .......... .......... .......... 32% 50.7M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30500K .......... .......... .......... .......... .......... 32% 61.4M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30550K .......... .......... .......... .......... .......... 32% 129M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30600K .......... .......... .......... .......... .......... 32% 40.5M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30650K .......... .......... .......... .......... .......... 33% 57.3M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30700K .......... .......... .......... .......... .......... 33% 74.4M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30750K .......... .......... .......... .......... .......... 33% 77.0M 1s 2024-03-04T18:07:21.762 INFO:teuthology.orchestra.run.smithi047.stderr: 30800K .......... .......... .......... .......... .......... 33% 72.1M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 30850K .......... .......... .......... .......... .......... 33% 52.6M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 30900K .......... .......... .......... .......... .......... 33% 55.8M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 30950K .......... .......... .......... .......... .......... 33% 103M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31000K .......... .......... .......... .......... .......... 33% 114M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31050K .......... .......... .......... .......... .......... 33% 59.9M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31100K .......... .......... .......... .......... .......... 33% 63.7M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31150K .......... .......... .......... .......... .......... 33% 47.5M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31200K .......... .......... .......... .......... .......... 33% 46.7M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31250K .......... .......... .......... .......... .......... 33% 51.0M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31300K .......... .......... .......... .......... .......... 33% 103M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31350K .......... .......... .......... .......... .......... 33% 46.6M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31400K .......... .......... .......... .......... .......... 33% 44.3M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31450K .......... .......... .......... .......... .......... 33% 50.8M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31500K .......... .......... .......... .......... .......... 33% 54.7M 1s 2024-03-04T18:07:21.763 INFO:teuthology.orchestra.run.smithi047.stderr: 31550K .......... .......... .......... .......... .......... 34% 113M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 31600K .......... .......... .......... .......... .......... 34% 46.7M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 31650K .......... .......... .......... .......... .......... 34% 50.1M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 31700K .......... .......... .......... .......... .......... 34% 48.7M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 31750K .......... .......... .......... .......... .......... 34% 121M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 31800K .......... .......... .......... .......... .......... 34% 50.2M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 31850K .......... .......... .......... .......... .......... 34% 39.5M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 31900K .......... .......... .......... .......... .......... 34% 129M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 31950K .......... .......... .......... .......... .......... 34% 44.8M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 32000K .......... .......... .......... .......... .......... 34% 115M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 32050K .......... .......... .......... .......... .......... 34% 54.3M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 32100K .......... .......... .......... .......... .......... 34% 81.6M 1s 2024-03-04T18:07:21.764 INFO:teuthology.orchestra.run.smithi047.stderr: 32150K .......... .......... .......... .......... .......... 34% 117M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32200K .......... .......... .......... .......... .......... 34% 108M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32250K .......... .......... .......... .......... .......... 34% 40.9M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32300K .......... .......... .......... .......... .......... 34% 40.5M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32350K .......... .......... .......... .......... .......... 34% 119M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32400K .......... .......... .......... .......... .......... 34% 56.8M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32450K .......... .......... .......... .......... .......... 34% 104M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32500K .......... .......... .......... .......... .......... 35% 77.5M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32550K .......... .......... .......... .......... .......... 35% 33.9M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32600K .......... .......... .......... .......... .......... 35% 54.6M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32650K .......... .......... .......... .......... .......... 35% 87.3M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32700K .......... .......... .......... .......... .......... 35% 118M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32750K .......... .......... .......... .......... .......... 35% 35.3M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32800K .......... .......... .......... .......... .......... 35% 64.3M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32850K .......... .......... .......... .......... .......... 35% 80.7M 1s 2024-03-04T18:07:21.765 INFO:teuthology.orchestra.run.smithi047.stderr: 32900K .......... .......... .......... .......... .......... 35% 130M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 32950K .......... .......... .......... .......... .......... 35% 121M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33000K .......... .......... .......... .......... .......... 35% 32.4M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33050K .......... .......... .......... .......... .......... 35% 111M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33100K .......... .......... .......... .......... .......... 35% 66.7M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33150K .......... .......... .......... .......... .......... 35% 117M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33200K .......... .......... .......... .......... .......... 35% 79.8M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33250K .......... .......... .......... .......... .......... 35% 34.9M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33300K .......... .......... .......... .......... .......... 35% 103M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33350K .......... .......... .......... .......... .......... 35% 119M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33400K .......... .......... .......... .......... .......... 36% 44.5M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33450K .......... .......... .......... .......... .......... 36% 57.8M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33500K .......... .......... .......... .......... .......... 36% 84.2M 1s 2024-03-04T18:07:21.766 INFO:teuthology.orchestra.run.smithi047.stderr: 33550K .......... .......... .......... .......... .......... 36% 48.1M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 33600K .......... .......... .......... .......... .......... 36% 112M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 33650K .......... .......... .......... .......... .......... 36% 48.9M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 33700K .......... .......... .......... .......... .......... 36% 112M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 33750K .......... .......... .......... .......... .......... 36% 96.1M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 33800K .......... .......... .......... .......... .......... 36% 64.8M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 33850K .......... .......... .......... .......... .......... 36% 115M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 33900K .......... .......... .......... .......... .......... 36% 47.9M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 33950K .......... .......... .......... .......... .......... 36% 74.5M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 34000K .......... .......... .......... .......... .......... 36% 45.6M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 34050K .......... .......... .......... .......... .......... 36% 122M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 34100K .......... .......... .......... .......... .......... 36% 55.8M 1s 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr: 34150K .......... .......... .......... .......... .......... 36% 67.6M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34200K .......... .......... .......... .......... .......... 36% 45.6M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34250K .......... .......... .......... .......... .......... 36% 47.2M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34300K .......... .......... .......... .......... .......... 36% 106M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34350K .......... .......... .......... .......... .......... 37% 86.3M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34400K .......... .......... .......... .......... .......... 37% 44.9M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34450K .......... .......... .......... .......... .......... 37% 54.9M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34500K .......... .......... .......... .......... .......... 37% 111M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34550K .......... .......... .......... .......... .......... 37% 89.5M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34600K .......... .......... .......... .......... .......... 37% 67.5M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34650K .......... .......... .......... .......... .......... 37% 30.3M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34700K .......... .......... .......... .......... .......... 37% 99.0M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34750K .......... .......... .......... .......... .......... 37% 114M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34800K .......... .......... .......... .......... .......... 37% 49.2M 1s 2024-03-04T18:07:21.768 INFO:teuthology.orchestra.run.smithi047.stderr: 34850K .......... .......... .......... .......... .......... 37% 71.7M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 34900K .......... .......... .......... .......... .......... 37% 43.6M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 34950K .......... .......... .......... .......... .......... 37% 82.5M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35000K .......... .......... .......... .......... .......... 37% 112M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35050K .......... .......... .......... .......... .......... 37% 56.5M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35100K .......... .......... .......... .......... .......... 37% 44.5M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35150K .......... .......... .......... .......... .......... 37% 94.8M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35200K .......... .......... .......... .......... .......... 37% 55.5M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35250K .......... .......... .......... .......... .......... 37% 107M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35300K .......... .......... .......... .......... .......... 38% 59.9M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35350K .......... .......... .......... .......... .......... 38% 37.0M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35400K .......... .......... .......... .......... .......... 38% 90.9M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35450K .......... .......... .......... .......... .......... 38% 61.5M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35500K .......... .......... .......... .......... .......... 38% 100M 1s 2024-03-04T18:07:21.769 INFO:teuthology.orchestra.run.smithi047.stderr: 35550K .......... .......... .......... .......... .......... 38% 37.2M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 35600K .......... .......... .......... .......... .......... 38% 68.0M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 35650K .......... .......... .......... .......... .......... 38% 92.8M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 35700K .......... .......... .......... .......... .......... 38% 62.6M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 35750K .......... .......... .......... .......... .......... 38% 94.5M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 35800K .......... .......... .......... .......... .......... 38% 24.3M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 35850K .......... .......... .......... .......... .......... 38% 57.7M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 35900K .......... .......... .......... .......... .......... 38% 76.7M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 35950K .......... .......... .......... .......... .......... 38% 111M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 36000K .......... .......... .......... .......... .......... 38% 61.5M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 36050K .......... .......... .......... .......... .......... 38% 34.4M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 36100K .......... .......... .......... .......... .......... 38% 83.2M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 36150K .......... .......... .......... .......... .......... 38% 46.7M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 36200K .......... .......... .......... .......... .......... 39% 103M 1s 2024-03-04T18:07:21.770 INFO:teuthology.orchestra.run.smithi047.stderr: 36250K .......... .......... .......... .......... .......... 39% 42.7M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36300K .......... .......... .......... .......... .......... 39% 96.8M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36350K .......... .......... .......... .......... .......... 39% 50.8M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36400K .......... .......... .......... .......... .......... 39% 103M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36450K .......... .......... .......... .......... .......... 39% 44.3M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36500K .......... .......... .......... .......... .......... 39% 63.5M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36550K .......... .......... .......... .......... .......... 39% 72.6M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36600K .......... .......... .......... .......... .......... 39% 38.9M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36650K .......... .......... .......... .......... .......... 39% 108M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36700K .......... .......... .......... .......... .......... 39% 66.0M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36750K .......... .......... .......... .......... .......... 39% 57.4M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36800K .......... .......... .......... .......... .......... 39% 61.5M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36850K .......... .......... .......... .......... .......... 39% 60.1M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36900K .......... .......... .......... .......... .......... 39% 80.6M 1s 2024-03-04T18:07:21.771 INFO:teuthology.orchestra.run.smithi047.stderr: 36950K .......... .......... .......... .......... .......... 39% 76.6M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37000K .......... .......... .......... .......... .......... 39% 57.2M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37050K .......... .......... .......... .......... .......... 39% 45.0M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37100K .......... .......... .......... .......... .......... 39% 102M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37150K .......... .......... .......... .......... .......... 40% 108M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37200K .......... .......... .......... .......... .......... 40% 53.0M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37250K .......... .......... .......... .......... .......... 40% 46.0M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37300K .......... .......... .......... .......... .......... 40% 77.2M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37350K .......... .......... .......... .......... .......... 40% 109M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37400K .......... .......... .......... .......... .......... 40% 63.8M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37450K .......... .......... .......... .......... .......... 40% 74.0M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37500K .......... .......... .......... .......... .......... 40% 89.0M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37550K .......... .......... .......... .......... .......... 40% 53.8M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37600K .......... .......... .......... .......... .......... 40% 90.8M 1s 2024-03-04T18:07:21.772 INFO:teuthology.orchestra.run.smithi047.stderr: 37650K .......... .......... .......... .......... .......... 40% 70.3M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 37700K .......... .......... .......... .......... .......... 40% 39.6M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 37750K .......... .......... .......... .......... .......... 40% 60.4M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 37800K .......... .......... .......... .......... .......... 40% 110M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 37850K .......... .......... .......... .......... .......... 40% 77.4M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 37900K .......... .......... .......... .......... .......... 40% 46.3M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 37950K .......... .......... .......... .......... .......... 40% 62.1M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 38000K .......... .......... .......... .......... .......... 40% 71.6M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 38050K .......... .......... .......... .......... .......... 41% 108M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 38100K .......... .......... .......... .......... .......... 41% 61.2M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 38150K .......... .......... .......... .......... .......... 41% 49.7M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 38200K .......... .......... .......... .......... .......... 41% 93.3M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 38250K .......... .......... .......... .......... .......... 41% 103M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 38300K .......... .......... .......... .......... .......... 41% 77.5M 1s 2024-03-04T18:07:21.773 INFO:teuthology.orchestra.run.smithi047.stderr: 38350K .......... .......... .......... .......... .......... 41% 42.0M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38400K .......... .......... .......... .......... .......... 41% 61.1M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38450K .......... .......... .......... .......... .......... 41% 97.7M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38500K .......... .......... .......... .......... .......... 41% 97.1M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38550K .......... .......... .......... .......... .......... 41% 107M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38600K .......... .......... .......... .......... .......... 41% 29.3M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38650K .......... .......... .......... .......... .......... 41% 96.6M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38700K .......... .......... .......... .......... .......... 41% 76.4M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38750K .......... .......... .......... .......... .......... 41% 111M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38800K .......... .......... .......... .......... .......... 41% 94.6M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38850K .......... .......... .......... .......... .......... 41% 31.0M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38900K .......... .......... .......... .......... .......... 41% 76.0M 1s 2024-03-04T18:07:21.774 INFO:teuthology.orchestra.run.smithi047.stderr: 38950K .......... .......... .......... .......... .......... 41% 98.2M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39000K .......... .......... .......... .......... .......... 42% 27.8M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39050K .......... .......... .......... .......... .......... 42% 48.1M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39100K .......... .......... .......... .......... .......... 42% 107M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39150K .......... .......... .......... .......... .......... 42% 52.3M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39200K .......... .......... .......... .......... .......... 42% 100M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39250K .......... .......... .......... .......... .......... 42% 49.2M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39300K .......... .......... .......... .......... .......... 42% 91.4M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39350K .......... .......... .......... .......... .......... 42% 62.0M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39400K .......... .......... .......... .......... .......... 42% 42.8M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39450K .......... .......... .......... .......... .......... 42% 95.4M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39500K .......... .......... .......... .......... .......... 42% 62.6M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39550K .......... .......... .......... .......... .......... 42% 75.7M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39600K .......... .......... .......... .......... .......... 42% 44.1M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39650K .......... .......... .......... .......... .......... 42% 96.0M 1s 2024-03-04T18:07:21.775 INFO:teuthology.orchestra.run.smithi047.stderr: 39700K .......... .......... .......... .......... .......... 42% 46.6M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 39750K .......... .......... .......... .......... .......... 42% 46.0M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 39800K .......... .......... .......... .......... .......... 42% 80.8M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 39850K .......... .......... .......... .......... .......... 42% 48.8M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 39900K .......... .......... .......... .......... .......... 43% 83.6M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 39950K .......... .......... .......... .......... .......... 43% 99.8M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 40000K .......... .......... .......... .......... .......... 43% 44.2M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 40050K .......... .......... .......... .......... .......... 43% 47.0M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 40100K .......... .......... .......... .......... .......... 43% 78.3M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 40150K .......... .......... .......... .......... .......... 43% 101M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 40200K .......... .......... .......... .......... .......... 43% 58.2M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 40250K .......... .......... .......... .......... .......... 43% 105M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 40300K .......... .......... .......... .......... .......... 43% 67.9M 1s 2024-03-04T18:07:21.776 INFO:teuthology.orchestra.run.smithi047.stderr: 40350K .......... .......... .......... .......... .......... 43% 99.2M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40400K .......... .......... .......... .......... .......... 43% 80.6M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40450K .......... .......... .......... .......... .......... 43% 35.9M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40500K .......... .......... .......... .......... .......... 43% 71.3M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40550K .......... .......... .......... .......... .......... 43% 67.7M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40600K .......... .......... .......... .......... .......... 43% 57.9M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40650K .......... .......... .......... .......... .......... 43% 70.6M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40700K .......... .......... .......... .......... .......... 43% 67.5M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40750K .......... .......... .......... .......... .......... 43% 46.6M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40800K .......... .......... .......... .......... .......... 43% 63.6M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40850K .......... .......... .......... .......... .......... 44% 72.1M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40900K .......... .......... .......... .......... .......... 44% 63.8M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 40950K .......... .......... .......... .......... .......... 44% 71.2M 1s 2024-03-04T18:07:21.777 INFO:teuthology.orchestra.run.smithi047.stderr: 41000K .......... .......... .......... .......... .......... 44% 53.0M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41050K .......... .......... .......... .......... .......... 44% 61.6M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41100K .......... .......... .......... .......... .......... 44% 68.7M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41150K .......... .......... .......... .......... .......... 44% 73.5M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41200K .......... .......... .......... .......... .......... 44% 60.8M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41250K .......... .......... .......... .......... .......... 44% 85.5M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41300K .......... .......... .......... .......... .......... 44% 113M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41350K .......... .......... .......... .......... .......... 44% 130M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41400K .......... .......... .......... .......... .......... 44% 41.5M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41450K .......... .......... .......... .......... .......... 44% 62.1M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41500K .......... .......... .......... .......... .......... 44% 64.9M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41550K .......... .......... .......... .......... .......... 44% 70.5M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41600K .......... .......... .......... .......... .......... 44% 45.9M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41650K .......... .......... .......... .......... .......... 44% 115M 1s 2024-03-04T18:07:21.778 INFO:teuthology.orchestra.run.smithi047.stderr: 41700K .......... .......... .......... .......... .......... 44% 123M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 41750K .......... .......... .......... .......... .......... 44% 117M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 41800K .......... .......... .......... .......... .......... 45% 58.7M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 41850K .......... .......... .......... .......... .......... 45% 55.4M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 41900K .......... .......... .......... .......... .......... 45% 46.4M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 41950K .......... .......... .......... .......... .......... 45% 63.0M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 42000K .......... .......... .......... .......... .......... 45% 98.4M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 42050K .......... .......... .......... .......... .......... 45% 35.1M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 42100K .......... .......... .......... .......... .......... 45% 107M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 42150K .......... .......... .......... .......... .......... 45% 45.5M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 42200K .......... .......... .......... .......... .......... 45% 48.3M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 42250K .......... .......... .......... .......... .......... 45% 97.7M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 42300K .......... .......... .......... .......... .......... 45% 121M 1s 2024-03-04T18:07:21.779 INFO:teuthology.orchestra.run.smithi047.stderr: 42350K .......... .......... .......... .......... .......... 45% 63.3M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42400K .......... .......... .......... .......... .......... 45% 55.8M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42450K .......... .......... .......... .......... .......... 45% 109M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42500K .......... .......... .......... .......... .......... 45% 106M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42550K .......... .......... .......... .......... .......... 45% 12.7M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42600K .......... .......... .......... .......... .......... 45% 102M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42650K .......... .......... .......... .......... .......... 45% 127M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42700K .......... .......... .......... .......... .......... 46% 120M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42750K .......... .......... .......... .......... .......... 46% 125M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42800K .......... .......... .......... .......... .......... 46% 126M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42850K .......... .......... .......... .......... .......... 46% 122M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42900K .......... .......... .......... .......... .......... 46% 82.4M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 42950K .......... .......... .......... .......... .......... 46% 95.9M 1s 2024-03-04T18:07:21.780 INFO:teuthology.orchestra.run.smithi047.stderr: 43000K .......... .......... .......... .......... .......... 46% 71.6M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43050K .......... .......... .......... .......... .......... 46% 59.3M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43100K .......... .......... .......... .......... .......... 46% 61.5M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43150K .......... .......... .......... .......... .......... 46% 33.3M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43200K .......... .......... .......... .......... .......... 46% 100M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43250K .......... .......... .......... .......... .......... 46% 48.6M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43300K .......... .......... .......... .......... .......... 46% 38.5M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43350K .......... .......... .......... .......... .......... 46% 49.8M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43400K .......... .......... .......... .......... .......... 46% 71.4M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43450K .......... .......... .......... .......... .......... 46% 102M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43500K .......... .......... .......... .......... .......... 46% 51.3M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43550K .......... .......... .......... .......... .......... 46% 48.2M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43600K .......... .......... .......... .......... .......... 46% 85.0M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43650K .......... .......... .......... .......... .......... 47% 123M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43700K .......... .......... .......... .......... .......... 47% 38.5M 1s 2024-03-04T18:07:21.781 INFO:teuthology.orchestra.run.smithi047.stderr: 43750K .......... .......... .......... .......... .......... 47% 92.1M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 43800K .......... .......... .......... .......... .......... 47% 68.8M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 43850K .......... .......... .......... .......... .......... 47% 40.8M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 43900K .......... .......... .......... .......... .......... 47% 111M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 43950K .......... .......... .......... .......... .......... 47% 73.6M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44000K .......... .......... .......... .......... .......... 47% 72.1M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44050K .......... .......... .......... .......... .......... 47% 79.6M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44100K .......... .......... .......... .......... .......... 47% 97.0M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44150K .......... .......... .......... .......... .......... 47% 23.5M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44200K .......... .......... .......... .......... .......... 47% 63.5M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44250K .......... .......... .......... .......... .......... 47% 101M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44300K .......... .......... .......... .......... .......... 47% 65.2M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44350K .......... .......... .......... .......... .......... 47% 106M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44400K .......... .......... .......... .......... .......... 47% 48.9M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44450K .......... .......... .......... .......... .......... 47% 99.8M 1s 2024-03-04T18:07:21.782 INFO:teuthology.orchestra.run.smithi047.stderr: 44500K .......... .......... .......... .......... .......... 47% 78.8M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 44550K .......... .......... .......... .......... .......... 48% 52.6M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 44600K .......... .......... .......... .......... .......... 48% 111M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 44650K .......... .......... .......... .......... .......... 48% 54.9M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 44700K .......... .......... .......... .......... .......... 48% 114M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 44750K .......... .......... .......... .......... .......... 48% 49.8M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 44800K .......... .......... .......... .......... .......... 48% 112M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 44850K .......... .......... .......... .......... .......... 48% 51.0M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 44900K .......... .......... .......... .......... .......... 48% 69.4M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 44950K .......... .......... .......... .......... .......... 48% 46.4M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 45000K .......... .......... .......... .......... .......... 48% 46.3M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 45050K .......... .......... .......... .......... .......... 48% 112M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 45100K .......... .......... .......... .......... .......... 48% 58.0M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 45150K .......... .......... .......... .......... .......... 48% 60.7M 1s 2024-03-04T18:07:21.783 INFO:teuthology.orchestra.run.smithi047.stderr: 45200K .......... .......... .......... .......... .......... 48% 74.8M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45250K .......... .......... .......... .......... .......... 48% 115M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45300K .......... .......... .......... .......... .......... 48% 57.6M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45350K .......... .......... .......... .......... .......... 48% 90.8M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45400K .......... .......... .......... .......... .......... 48% 96.7M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45450K .......... .......... .......... .......... .......... 48% 110M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45500K .......... .......... .......... .......... .......... 49% 117M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45550K .......... .......... .......... .......... .......... 49% 13.3M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45600K .......... .......... .......... .......... .......... 49% 120M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45650K .......... .......... .......... .......... .......... 49% 66.2M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45700K .......... .......... .......... .......... .......... 49% 97.6M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45750K .......... .......... .......... .......... .......... 49% 57.7M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45800K .......... .......... .......... .......... .......... 49% 126M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45850K .......... .......... .......... .......... .......... 49% 53.0M 1s 2024-03-04T18:07:21.784 INFO:teuthology.orchestra.run.smithi047.stderr: 45900K .......... .......... .......... .......... .......... 49% 115M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 45950K .......... .......... .......... .......... .......... 49% 105M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46000K .......... .......... .......... .......... .......... 49% 67.2M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46050K .......... .......... .......... .......... .......... 49% 78.3M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46100K .......... .......... .......... .......... .......... 49% 104M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46150K .......... .......... .......... .......... .......... 49% 118M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46200K .......... .......... .......... .......... .......... 49% 65.1M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46250K .......... .......... .......... .......... .......... 49% 105M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46300K .......... .......... .......... .......... .......... 49% 46.9M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46350K .......... .......... .......... .......... .......... 49% 50.1M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46400K .......... .......... .......... .......... .......... 49% 92.1M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46450K .......... .......... .......... .......... .......... 50% 93.0M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46500K .......... .......... .......... .......... .......... 50% 114M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46550K .......... .......... .......... .......... .......... 50% 36.6M 1s 2024-03-04T18:07:21.785 INFO:teuthology.orchestra.run.smithi047.stderr: 46600K .......... .......... .......... .......... .......... 50% 49.3M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 46650K .......... .......... .......... .......... .......... 50% 29.2M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 46700K .......... .......... .......... .......... .......... 50% 110M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 46750K .......... .......... .......... .......... .......... 50% 107M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 46800K .......... .......... .......... .......... .......... 50% 70.7M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 46850K .......... .......... .......... .......... .......... 50% 49.2M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 46900K .......... .......... .......... .......... .......... 50% 38.0M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 46950K .......... .......... .......... .......... .......... 50% 108M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 47000K .......... .......... .......... .......... .......... 50% 105M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 47050K .......... .......... .......... .......... .......... 50% 65.6M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 47100K .......... .......... .......... .......... .......... 50% 31.8M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 47150K .......... .......... .......... .......... .......... 50% 126M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 47200K .......... .......... .......... .......... .......... 50% 93.9M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 47250K .......... .......... .......... .......... .......... 50% 114M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 47300K .......... .......... .......... .......... .......... 50% 54.5M 1s 2024-03-04T18:07:21.786 INFO:teuthology.orchestra.run.smithi047.stderr: 47350K .......... .......... .......... .......... .......... 51% 39.4M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47400K .......... .......... .......... .......... .......... 51% 74.2M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47450K .......... .......... .......... .......... .......... 51% 51.0M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47500K .......... .......... .......... .......... .......... 51% 40.4M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47550K .......... .......... .......... .......... .......... 51% 50.3M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47600K .......... .......... .......... .......... .......... 51% 24.8M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47650K .......... .......... .......... .......... .......... 51% 103M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47700K .......... .......... .......... .......... .......... 51% 33.7M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47750K .......... .......... .......... .......... .......... 51% 44.7M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47800K .......... .......... .......... .......... .......... 51% 57.6M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47850K .......... .......... .......... .......... .......... 51% 104M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47900K .......... .......... .......... .......... .......... 51% 57.7M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 47950K .......... .......... .......... .......... .......... 51% 54.2M 1s 2024-03-04T18:07:21.787 INFO:teuthology.orchestra.run.smithi047.stderr: 48000K .......... .......... .......... .......... .......... 51% 95.8M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48050K .......... .......... .......... .......... .......... 51% 47.2M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48100K .......... .......... .......... .......... .......... 51% 105M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48150K .......... .......... .......... .......... .......... 51% 97.4M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48200K .......... .......... .......... .......... .......... 51% 53.6M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48250K .......... .......... .......... .......... .......... 51% 52.7M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48300K .......... .......... .......... .......... .......... 52% 112M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48350K .......... .......... .......... .......... .......... 52% 35.3M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48400K .......... .......... .......... .......... .......... 52% 48.4M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48450K .......... .......... .......... .......... .......... 52% 63.1M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48500K .......... .......... .......... .......... .......... 52% 17.3M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48550K .......... .......... .......... .......... .......... 52% 90.7M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48600K .......... .......... .......... .......... .......... 52% 54.3M 1s 2024-03-04T18:07:21.788 INFO:teuthology.orchestra.run.smithi047.stderr: 48650K .......... .......... .......... .......... .......... 52% 61.3M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 48700K .......... .......... .......... .......... .......... 52% 71.5M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 48750K .......... .......... .......... .......... .......... 52% 85.3M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 48800K .......... .......... .......... .......... .......... 52% 123M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 48850K .......... .......... .......... .......... .......... 52% 116M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 48900K .......... .......... .......... .......... .......... 52% 31.1M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 48950K .......... .......... .......... .......... .......... 52% 97.3M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 49000K .......... .......... .......... .......... .......... 52% 121M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 49050K .......... .......... .......... .......... .......... 52% 48.8M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 49100K .......... .......... .......... .......... .......... 52% 48.2M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 49150K .......... .......... .......... .......... .......... 52% 88.7M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 49200K .......... .......... .......... .......... .......... 53% 119M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 49250K .......... .......... .......... .......... .......... 53% 102M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 49300K .......... .......... .......... .......... .......... 53% 60.0M 1s 2024-03-04T18:07:21.789 INFO:teuthology.orchestra.run.smithi047.stderr: 49350K .......... .......... .......... .......... .......... 53% 53.6M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49400K .......... .......... .......... .......... .......... 53% 65.2M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49450K .......... .......... .......... .......... .......... 53% 110M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49500K .......... .......... .......... .......... .......... 53% 56.3M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49550K .......... .......... .......... .......... .......... 53% 55.9M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49600K .......... .......... .......... .......... .......... 53% 77.4M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49650K .......... .......... .......... .......... .......... 53% 85.1M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49700K .......... .......... .......... .......... .......... 53% 46.6M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49750K .......... .......... .......... .......... .......... 53% 62.6M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49800K .......... .......... .......... .......... .......... 53% 113M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49850K .......... .......... .......... .......... .......... 53% 56.4M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49900K .......... .......... .......... .......... .......... 53% 52.7M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 49950K .......... .......... .......... .......... .......... 53% 58.5M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 50000K .......... .......... .......... .......... .......... 53% 115M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 50050K .......... .......... .......... .......... .......... 53% 109M 1s 2024-03-04T18:07:21.790 INFO:teuthology.orchestra.run.smithi047.stderr: 50100K .......... .......... .......... .......... .......... 53% 79.1M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50150K .......... .......... .......... .......... .......... 54% 36.6M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50200K .......... .......... .......... .......... .......... 54% 105M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50250K .......... .......... .......... .......... .......... 54% 124M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50300K .......... .......... .......... .......... .......... 54% 125M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50350K .......... .......... .......... .......... .......... 54% 48.0M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50400K .......... .......... .......... .......... .......... 54% 126M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50450K .......... .......... .......... .......... .......... 54% 111M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50500K .......... .......... .......... .......... .......... 54% 41.6M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50550K .......... .......... .......... .......... .......... 54% 71.5M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50600K .......... .......... .......... .......... .......... 54% 110M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50650K .......... .......... .......... .......... .......... 54% 30.3M 1s 2024-03-04T18:07:21.791 INFO:teuthology.orchestra.run.smithi047.stderr: 50700K .......... .......... .......... .......... .......... 54% 89.9M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 50750K .......... .......... .......... .......... .......... 54% 47.0M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 50800K .......... .......... .......... .......... .......... 54% 49.3M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 50850K .......... .......... .......... .......... .......... 54% 65.6M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 50900K .......... .......... .......... .......... .......... 54% 118M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 50950K .......... .......... .......... .......... .......... 54% 59.0M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 51000K .......... .......... .......... .......... .......... 54% 55.7M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 51050K .......... .......... .......... .......... .......... 55% 70.4M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 51100K .......... .......... .......... .......... .......... 55% 38.0M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 51150K .......... .......... .......... .......... .......... 55% 110M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 51200K .......... .......... .......... .......... .......... 55% 75.2M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 51250K .......... .......... .......... .......... .......... 55% 54.9M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 51300K .......... .......... .......... .......... .......... 55% 53.4M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 51350K .......... .......... .......... .......... .......... 55% 121M 1s 2024-03-04T18:07:21.792 INFO:teuthology.orchestra.run.smithi047.stderr: 51400K .......... .......... .......... .......... .......... 55% 47.9M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51450K .......... .......... .......... .......... .......... 55% 35.4M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51500K .......... .......... .......... .......... .......... 55% 35.4M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51550K .......... .......... .......... .......... .......... 55% 30.0M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51600K .......... .......... .......... .......... .......... 55% 75.5M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51650K .......... .......... .......... .......... .......... 55% 34.7M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51700K .......... .......... .......... .......... .......... 55% 35.8M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51750K .......... .......... .......... .......... .......... 55% 50.3M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51800K .......... .......... .......... .......... .......... 55% 111M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51850K .......... .......... .......... .......... .......... 55% 46.7M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51900K .......... .......... .......... .......... .......... 55% 35.7M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 51950K .......... .......... .......... .......... .......... 55% 43.1M 1s 2024-03-04T18:07:21.793 INFO:teuthology.orchestra.run.smithi047.stderr: 52000K .......... .......... .......... .......... .......... 56% 90.9M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52050K .......... .......... .......... .......... .......... 56% 108M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52100K .......... .......... .......... .......... .......... 56% 36.9M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52150K .......... .......... .......... .......... .......... 56% 65.3M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52200K .......... .......... .......... .......... .......... 56% 66.6M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52250K .......... .......... .......... .......... .......... 56% 54.5M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52300K .......... .......... .......... .......... .......... 56% 110M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52350K .......... .......... .......... .......... .......... 56% 46.2M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52400K .......... .......... .......... .......... .......... 56% 74.7M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52450K .......... .......... .......... .......... .......... 56% 86.3M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52500K .......... .......... .......... .......... .......... 56% 120M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52550K .......... .......... .......... .......... .......... 56% 46.0M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52600K .......... .......... .......... .......... .......... 56% 65.8M 1s 2024-03-04T18:07:21.794 INFO:teuthology.orchestra.run.smithi047.stderr: 52650K .......... .......... .......... .......... .......... 56% 76.8M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 52700K .......... .......... .......... .......... .......... 56% 51.4M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 52750K .......... .......... .......... .......... .......... 56% 96.4M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 52800K .......... .......... .......... .......... .......... 56% 57.3M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 52850K .......... .......... .......... .......... .......... 56% 73.5M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 52900K .......... .......... .......... .......... .......... 56% 31.4M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 52950K .......... .......... .......... .......... .......... 57% 75.8M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 53000K .......... .......... .......... .......... .......... 57% 100M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 53050K .......... .......... .......... .......... .......... 57% 41.4M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 53100K .......... .......... .......... .......... .......... 57% 61.8M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 53150K .......... .......... .......... .......... .......... 57% 99.5M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 53200K .......... .......... .......... .......... .......... 57% 121M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 53250K .......... .......... .......... .......... .......... 57% 45.8M 1s 2024-03-04T18:07:21.795 INFO:teuthology.orchestra.run.smithi047.stderr: 53300K .......... .......... .......... .......... .......... 57% 69.1M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53350K .......... .......... .......... .......... .......... 57% 83.6M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53400K .......... .......... .......... .......... .......... 57% 64.9M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53450K .......... .......... .......... .......... .......... 57% 103M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53500K .......... .......... .......... .......... .......... 57% 39.9M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53550K .......... .......... .......... .......... .......... 57% 38.9M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53600K .......... .......... .......... .......... .......... 57% 94.9M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53650K .......... .......... .......... .......... .......... 57% 108M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53700K .......... .......... .......... .......... .......... 57% 37.5M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53750K .......... .......... .......... .......... .......... 57% 45.6M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53800K .......... .......... .......... .......... .......... 57% 80.6M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53850K .......... .......... .......... .......... .......... 58% 89.5M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53900K .......... .......... .......... .......... .......... 58% 34.6M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 53950K .......... .......... .......... .......... .......... 58% 108M 1s 2024-03-04T18:07:21.796 INFO:teuthology.orchestra.run.smithi047.stderr: 54000K .......... .......... .......... .......... .......... 58% 60.3M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54050K .......... .......... .......... .......... .......... 58% 51.5M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54100K .......... .......... .......... .......... .......... 58% 44.4M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54150K .......... .......... .......... .......... .......... 58% 46.3M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54200K .......... .......... .......... .......... .......... 58% 101M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54250K .......... .......... .......... .......... .......... 58% 50.6M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54300K .......... .......... .......... .......... .......... 58% 111M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54350K .......... .......... .......... .......... .......... 58% 49.1M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54400K .......... .......... .......... .......... .......... 58% 110M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54450K .......... .......... .......... .......... .......... 58% 119M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54500K .......... .......... .......... .......... .......... 58% 60.3M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54550K .......... .......... .......... .......... .......... 58% 44.2M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54600K .......... .......... .......... .......... .......... 58% 67.9M 1s 2024-03-04T18:07:21.797 INFO:teuthology.orchestra.run.smithi047.stderr: 54650K .......... .......... .......... .......... .......... 58% 46.9M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 54700K .......... .......... .......... .......... .......... 58% 101M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 54750K .......... .......... .......... .......... .......... 58% 49.6M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 54800K .......... .......... .......... .......... .......... 59% 71.0M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 54850K .......... .......... .......... .......... .......... 59% 57.1M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 54900K .......... .......... .......... .......... .......... 59% 111M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 54950K .......... .......... .......... .......... .......... 59% 41.6M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 55000K .......... .......... .......... .......... .......... 59% 69.2M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 55050K .......... .......... .......... .......... .......... 59% 52.3M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 55100K .......... .......... .......... .......... .......... 59% 97.6M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 55150K .......... .......... .......... .......... .......... 59% 51.5M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 55200K .......... .......... .......... .......... .......... 59% 95.1M 1s 2024-03-04T18:07:21.798 INFO:teuthology.orchestra.run.smithi047.stderr: 55250K .......... .......... .......... .......... .......... 59% 76.1M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55300K .......... .......... .......... .......... .......... 59% 34.0M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55350K .......... .......... .......... .......... .......... 59% 113M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55400K .......... .......... .......... .......... .......... 59% 107M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55450K .......... .......... .......... .......... .......... 59% 59.6M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55500K .......... .......... .......... .......... .......... 59% 38.2M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55550K .......... .......... .......... .......... .......... 59% 121M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55600K .......... .......... .......... .......... .......... 59% 102M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55650K .......... .......... .......... .......... .......... 59% 65.4M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55700K .......... .......... .......... .......... .......... 60% 45.0M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55750K .......... .......... .......... .......... .......... 60% 28.8M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55800K .......... .......... .......... .......... .......... 60% 100M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55850K .......... .......... .......... .......... .......... 60% 45.5M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55900K .......... .......... .......... .......... .......... 60% 47.0M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 55950K .......... .......... .......... .......... .......... 60% 33.3M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 56000K .......... .......... .......... .......... .......... 60% 50.6M 1s 2024-03-04T18:07:21.799 INFO:teuthology.orchestra.run.smithi047.stderr: 56050K .......... .......... .......... .......... .......... 60% 101M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56100K .......... .......... .......... .......... .......... 60% 63.2M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56150K .......... .......... .......... .......... .......... 60% 58.4M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56200K .......... .......... .......... .......... .......... 60% 67.8M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56250K .......... .......... .......... .......... .......... 60% 118M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56300K .......... .......... .......... .......... .......... 60% 38.9M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56350K .......... .......... .......... .......... .......... 60% 42.7M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56400K .......... .......... .......... .......... .......... 60% 102M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56450K .......... .......... .......... .......... .......... 60% 35.8M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56500K .......... .......... .......... .......... .......... 60% 111M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56550K .......... .......... .......... .......... .......... 60% 47.2M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56600K .......... .......... .......... .......... .......... 60% 85.9M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56650K .......... .......... .......... .......... .......... 61% 81.0M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56700K .......... .......... .......... .......... .......... 61% 35.2M 1s 2024-03-04T18:07:21.800 INFO:teuthology.orchestra.run.smithi047.stderr: 56750K .......... .......... .......... .......... .......... 61% 115M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 56800K .......... .......... .......... .......... .......... 61% 64.9M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 56850K .......... .......... .......... .......... .......... 61% 65.5M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 56900K .......... .......... .......... .......... .......... 61% 72.0M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 56950K .......... .......... .......... .......... .......... 61% 117M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 57000K .......... .......... .......... .......... .......... 61% 40.5M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 57050K .......... .......... .......... .......... .......... 61% 64.4M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 57100K .......... .......... .......... .......... .......... 61% 55.0M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 57150K .......... .......... .......... .......... .......... 61% 67.0M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 57200K .......... .......... .......... .......... .......... 61% 109M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 57250K .......... .......... .......... .......... .......... 61% 70.5M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 57300K .......... .......... .......... .......... .......... 61% 58.1M 1s 2024-03-04T18:07:21.801 INFO:teuthology.orchestra.run.smithi047.stderr: 57350K .......... .......... .......... .......... .......... 61% 95.9M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57400K .......... .......... .......... .......... .......... 61% 119M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57450K .......... .......... .......... .......... .......... 61% 38.9M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57500K .......... .......... .......... .......... .......... 61% 96.1M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57550K .......... .......... .......... .......... .......... 62% 42.7M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57600K .......... .......... .......... .......... .......... 62% 81.5M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57650K .......... .......... .......... .......... .......... 62% 110M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57700K .......... .......... .......... .......... .......... 62% 37.5M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57750K .......... .......... .......... .......... .......... 62% 56.7M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57800K .......... .......... .......... .......... .......... 62% 58.9M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57850K .......... .......... .......... .......... .......... 62% 60.3M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57900K .......... .......... .......... .......... .......... 62% 76.4M 1s 2024-03-04T18:07:21.802 INFO:teuthology.orchestra.run.smithi047.stderr: 57950K .......... .......... .......... .......... .......... 62% 51.3M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58000K .......... .......... .......... .......... .......... 62% 70.2M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58050K .......... .......... .......... .......... .......... 62% 40.6M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58100K .......... .......... .......... .......... .......... 62% 80.8M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58150K .......... .......... .......... .......... .......... 62% 74.4M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58200K .......... .......... .......... .......... .......... 62% 52.1M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58250K .......... .......... .......... .......... .......... 62% 81.4M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58300K .......... .......... .......... .......... .......... 62% 69.3M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58350K .......... .......... .......... .......... .......... 62% 83.2M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58400K .......... .......... .......... .......... .......... 62% 60.3M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58450K .......... .......... .......... .......... .......... 62% 71.0M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58500K .......... .......... .......... .......... .......... 63% 87.7M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58550K .......... .......... .......... .......... .......... 63% 104M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58600K .......... .......... .......... .......... .......... 63% 44.5M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58650K .......... .......... .......... .......... .......... 63% 101M 1s 2024-03-04T18:07:21.803 INFO:teuthology.orchestra.run.smithi047.stderr: 58700K .......... .......... .......... .......... .......... 63% 105M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 58750K .......... .......... .......... .......... .......... 63% 22.1M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 58800K .......... .......... .......... .......... .......... 63% 24.7M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 58850K .......... .......... .......... .......... .......... 63% 99.9M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 58900K .......... .......... .......... .......... .......... 63% 25.1M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 58950K .......... .......... .......... .......... .......... 63% 55.2M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 59000K .......... .......... .......... .......... .......... 63% 31.7M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 59050K .......... .......... .......... .......... .......... 63% 77.2M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 59100K .......... .......... .......... .......... .......... 63% 92.8M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 59150K .......... .......... .......... .......... .......... 63% 41.6M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 59200K .......... .......... .......... .......... .......... 63% 95.5M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 59250K .......... .......... .......... .......... .......... 63% 64.5M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 59300K .......... .......... .......... .......... .......... 63% 123M 1s 2024-03-04T18:07:21.804 INFO:teuthology.orchestra.run.smithi047.stderr: 59350K .......... .......... .......... .......... .......... 63% 33.4M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59400K .......... .......... .......... .......... .......... 63% 55.1M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59450K .......... .......... .......... .......... .......... 64% 80.4M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59500K .......... .......... .......... .......... .......... 64% 37.9M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59550K .......... .......... .......... .......... .......... 64% 110M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59600K .......... .......... .......... .......... .......... 64% 59.3M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59650K .......... .......... .......... .......... .......... 64% 61.7M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59700K .......... .......... .......... .......... .......... 64% 77.8M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59750K .......... .......... .......... .......... .......... 64% 122M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59800K .......... .......... .......... .......... .......... 64% 37.2M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59850K .......... .......... .......... .......... .......... 64% 61.7M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59900K .......... .......... .......... .......... .......... 64% 78.7M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 59950K .......... .......... .......... .......... .......... 64% 53.6M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 60000K .......... .......... .......... .......... .......... 64% 116M 1s 2024-03-04T18:07:21.805 INFO:teuthology.orchestra.run.smithi047.stderr: 60050K .......... .......... .......... .......... .......... 64% 41.5M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60100K .......... .......... .......... .......... .......... 64% 90.8M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60150K .......... .......... .......... .......... .......... 64% 65.3M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60200K .......... .......... .......... .......... .......... 64% 41.9M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60250K .......... .......... .......... .......... .......... 64% 106M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60300K .......... .......... .......... .......... .......... 64% 101M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60350K .......... .......... .......... .......... .......... 65% 46.6M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60400K .......... .......... .......... .......... .......... 65% 91.8M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60450K .......... .......... .......... .......... .......... 65% 120M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60500K .......... .......... .......... .......... .......... 65% 38.2M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60550K .......... .......... .......... .......... .......... 65% 56.5M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60600K .......... .......... .......... .......... .......... 65% 66.7M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60650K .......... .......... .......... .......... .......... 65% 56.3M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60700K .......... .......... .......... .......... .......... 65% 106M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60750K .......... .......... .......... .......... .......... 65% 41.8M 1s 2024-03-04T18:07:21.806 INFO:teuthology.orchestra.run.smithi047.stderr: 60800K .......... .......... .......... .......... .......... 65% 68.8M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 60850K .......... .......... .......... .......... .......... 65% 65.2M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 60900K .......... .......... .......... .......... .......... 65% 41.5M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 60950K .......... .......... .......... .......... .......... 65% 95.6M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61000K .......... .......... .......... .......... .......... 65% 54.9M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61050K .......... .......... .......... .......... .......... 65% 86.1M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61100K .......... .......... .......... .......... .......... 65% 42.7M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61150K .......... .......... .......... .......... .......... 65% 116M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61200K .......... .......... .......... .......... .......... 65% 86.7M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61250K .......... .......... .......... .......... .......... 65% 65.8M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61300K .......... .......... .......... .......... .......... 66% 48.6M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61350K .......... .......... .......... .......... .......... 66% 50.7M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61400K .......... .......... .......... .......... .......... 66% 111M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61450K .......... .......... .......... .......... .......... 66% 104M 1s 2024-03-04T18:07:21.807 INFO:teuthology.orchestra.run.smithi047.stderr: 61500K .......... .......... .......... .......... .......... 66% 88.1M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 61550K .......... .......... .......... .......... .......... 66% 66.5M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 61600K .......... .......... .......... .......... .......... 66% 124M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 61650K .......... .......... .......... .......... .......... 66% 81.2M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 61700K .......... .......... .......... .......... .......... 66% 68.1M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 61750K .......... .......... .......... .......... .......... 66% 72.8M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 61800K .......... .......... .......... .......... .......... 66% 33.4M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 61850K .......... .......... .......... .......... .......... 66% 105M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 61900K .......... .......... .......... .......... .......... 66% 34.1M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 61950K .......... .......... .......... .......... .......... 66% 30.9M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 62000K .......... .......... .......... .......... .......... 66% 62.5M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 62050K .......... .......... .......... .......... .......... 66% 119M 1s 2024-03-04T18:07:21.808 INFO:teuthology.orchestra.run.smithi047.stderr: 62100K .......... .......... .......... .......... .......... 66% 23.6M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62150K .......... .......... .......... .......... .......... 66% 121M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62200K .......... .......... .......... .......... .......... 67% 62.9M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62250K .......... .......... .......... .......... .......... 67% 56.8M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62300K .......... .......... .......... .......... .......... 67% 110M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62350K .......... .......... .......... .......... .......... 67% 34.7M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62400K .......... .......... .......... .......... .......... 67% 23.9M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62450K .......... .......... .......... .......... .......... 67% 47.6M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62500K .......... .......... .......... .......... .......... 67% 44.0M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62550K .......... .......... .......... .......... .......... 67% 66.3M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62600K .......... .......... .......... .......... .......... 67% 108M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62650K .......... .......... .......... .......... .......... 67% 50.7M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62700K .......... .......... .......... .......... .......... 67% 123M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62750K .......... .......... .......... .......... .......... 67% 53.3M 1s 2024-03-04T18:07:21.809 INFO:teuthology.orchestra.run.smithi047.stderr: 62800K .......... .......... .......... .......... .......... 67% 90.9M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 62850K .......... .......... .......... .......... .......... 67% 51.3M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 62900K .......... .......... .......... .......... .......... 67% 105M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 62950K .......... .......... .......... .......... .......... 67% 45.0M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63000K .......... .......... .......... .......... .......... 67% 50.2M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63050K .......... .......... .......... .......... .......... 67% 106M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63100K .......... .......... .......... .......... .......... 67% 57.9M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63150K .......... .......... .......... .......... .......... 68% 81.4M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63200K .......... .......... .......... .......... .......... 68% 74.1M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63250K .......... .......... .......... .......... .......... 68% 40.5M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63300K .......... .......... .......... .......... .......... 68% 116M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63350K .......... .......... .......... .......... .......... 68% 71.2M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63400K .......... .......... .......... .......... .......... 68% 58.6M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63450K .......... .......... .......... .......... .......... 68% 61.0M 1s 2024-03-04T18:07:21.810 INFO:teuthology.orchestra.run.smithi047.stderr: 63500K .......... .......... .......... .......... .......... 68% 109M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 63550K .......... .......... .......... .......... .......... 68% 47.4M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 63600K .......... .......... .......... .......... .......... 68% 49.7M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 63650K .......... .......... .......... .......... .......... 68% 76.9M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 63700K .......... .......... .......... .......... .......... 68% 44.7M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 63750K .......... .......... .......... .......... .......... 68% 71.3M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 63800K .......... .......... .......... .......... .......... 68% 54.7M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 63850K .......... .......... .......... .......... .......... 68% 80.5M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 63900K .......... .......... .......... .......... .......... 68% 51.3M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 63950K .......... .......... .......... .......... .......... 68% 120M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 64000K .......... .......... .......... .......... .......... 68% 43.6M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 64050K .......... .......... .......... .......... .......... 68% 82.9M 1s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 64100K .......... .......... .......... .......... .......... 69% 71.0M 0s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 64150K .......... .......... .......... .......... .......... 69% 62.5M 0s 2024-03-04T18:07:21.811 INFO:teuthology.orchestra.run.smithi047.stderr: 64200K .......... .......... .......... .......... .......... 69% 112M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64250K .......... .......... .......... .......... .......... 69% 104M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64300K .......... .......... .......... .......... .......... 69% 60.7M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64350K .......... .......... .......... .......... .......... 69% 79.3M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64400K .......... .......... .......... .......... .......... 69% 93.9M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64450K .......... .......... .......... .......... .......... 69% 84.6M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64500K .......... .......... .......... .......... .......... 69% 122M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64550K .......... .......... .......... .......... .......... 69% 52.4M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64600K .......... .......... .......... .......... .......... 69% 99.7M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64650K .......... .......... .......... .......... .......... 69% 107M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64700K .......... .......... .......... .......... .......... 69% 58.8M 0s 2024-03-04T18:07:21.812 INFO:teuthology.orchestra.run.smithi047.stderr: 64750K .......... .......... .......... .......... .......... 69% 66.4M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 64800K .......... .......... .......... .......... .......... 69% 44.1M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 64850K .......... .......... .......... .......... .......... 69% 39.3M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 64900K .......... .......... .......... .......... .......... 69% 74.3M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 64950K .......... .......... .......... .......... .......... 69% 33.9M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 65000K .......... .......... .......... .......... .......... 70% 91.1M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 65050K .......... .......... .......... .......... .......... 70% 51.5M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 65100K .......... .......... .......... .......... .......... 70% 113M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 65150K .......... .......... .......... .......... .......... 70% 32.6M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 65200K .......... .......... .......... .......... .......... 70% 65.7M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 65250K .......... .......... .......... .......... .......... 70% 106M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 65300K .......... .......... .......... .......... .......... 70% 61.9M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 65350K .......... .......... .......... .......... .......... 70% 100M 0s 2024-03-04T18:07:21.813 INFO:teuthology.orchestra.run.smithi047.stderr: 65400K .......... .......... .......... .......... .......... 70% 37.6M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65450K .......... .......... .......... .......... .......... 70% 35.4M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65500K .......... .......... .......... .......... .......... 70% 58.4M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65550K .......... .......... .......... .......... .......... 70% 42.7M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65600K .......... .......... .......... .......... .......... 70% 121M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65650K .......... .......... .......... .......... .......... 70% 93.5M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65700K .......... .......... .......... .......... .......... 70% 66.8M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65750K .......... .......... .......... .......... .......... 70% 51.8M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65800K .......... .......... .......... .......... .......... 70% 48.1M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65850K .......... .......... .......... .......... .......... 70% 107M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65900K .......... .......... .......... .......... .......... 70% 73.6M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 65950K .......... .......... .......... .......... .......... 71% 54.7M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 66000K .......... .......... .......... .......... .......... 71% 92.1M 0s 2024-03-04T18:07:21.814 INFO:teuthology.orchestra.run.smithi047.stderr: 66050K .......... .......... .......... .......... .......... 71% 32.1M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66100K .......... .......... .......... .......... .......... 71% 111M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66150K .......... .......... .......... .......... .......... 71% 76.9M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66200K .......... .......... .......... .......... .......... 71% 77.1M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66250K .......... .......... .......... .......... .......... 71% 39.0M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66300K .......... .......... .......... .......... .......... 71% 104M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66350K .......... .......... .......... .......... .......... 71% 104M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66400K .......... .......... .......... .......... .......... 71% 60.0M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66450K .......... .......... .......... .......... .......... 71% 60.4M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66500K .......... .......... .......... .......... .......... 71% 123M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66550K .......... .......... .......... .......... .......... 71% 37.2M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66600K .......... .......... .......... .......... .......... 71% 51.1M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66650K .......... .......... .......... .......... .......... 71% 55.0M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66700K .......... .......... .......... .......... .......... 71% 78.5M 0s 2024-03-04T18:07:21.815 INFO:teuthology.orchestra.run.smithi047.stderr: 66750K .......... .......... .......... .......... .......... 71% 51.8M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 66800K .......... .......... .......... .......... .......... 71% 106M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 66850K .......... .......... .......... .......... .......... 72% 56.2M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 66900K .......... .......... .......... .......... .......... 72% 97.2M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 66950K .......... .......... .......... .......... .......... 72% 60.8M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 67000K .......... .......... .......... .......... .......... 72% 50.4M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 67050K .......... .......... .......... .......... .......... 72% 120M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 67100K .......... .......... .......... .......... .......... 72% 109M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 67150K .......... .......... .......... .......... .......... 72% 98.7M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 67200K .......... .......... .......... .......... .......... 72% 56.9M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 67250K .......... .......... .......... .......... .......... 72% 103M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 67300K .......... .......... .......... .......... .......... 72% 111M 0s 2024-03-04T18:07:21.816 INFO:teuthology.orchestra.run.smithi047.stderr: 67350K .......... .......... .......... .......... .......... 72% 71.6M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67400K .......... .......... .......... .......... .......... 72% 80.7M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67450K .......... .......... .......... .......... .......... 72% 58.8M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67500K .......... .......... .......... .......... .......... 72% 119M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67550K .......... .......... .......... .......... .......... 72% 43.7M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67600K .......... .......... .......... .......... .......... 72% 98.0M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67650K .......... .......... .......... .......... .......... 72% 42.9M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67700K .......... .......... .......... .......... .......... 72% 120M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67750K .......... .......... .......... .......... .......... 72% 50.4M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67800K .......... .......... .......... .......... .......... 73% 44.0M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67850K .......... .......... .......... .......... .......... 73% 80.5M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67900K .......... .......... .......... .......... .......... 73% 35.3M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 67950K .......... .......... .......... .......... .......... 73% 115M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 68000K .......... .......... .......... .......... .......... 73% 49.0M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 68050K .......... .......... .......... .......... .......... 73% 66.9M 0s 2024-03-04T18:07:21.817 INFO:teuthology.orchestra.run.smithi047.stderr: 68100K .......... .......... .......... .......... .......... 73% 35.4M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68150K .......... .......... .......... .......... .......... 73% 80.4M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68200K .......... .......... .......... .......... .......... 73% 91.7M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68250K .......... .......... .......... .......... .......... 73% 36.2M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68300K .......... .......... .......... .......... .......... 73% 77.6M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68350K .......... .......... .......... .......... .......... 73% 54.3M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68400K .......... .......... .......... .......... .......... 73% 80.8M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68450K .......... .......... .......... .......... .......... 73% 53.8M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68500K .......... .......... .......... .......... .......... 73% 65.8M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68550K .......... .......... .......... .......... .......... 73% 45.1M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68600K .......... .......... .......... .......... .......... 73% 77.0M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68650K .......... .......... .......... .......... .......... 73% 80.6M 0s 2024-03-04T18:07:21.818 INFO:teuthology.orchestra.run.smithi047.stderr: 68700K .......... .......... .......... .......... .......... 74% 41.8M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 68750K .......... .......... .......... .......... .......... 74% 92.4M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 68800K .......... .......... .......... .......... .......... 74% 33.4M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 68850K .......... .......... .......... .......... .......... 74% 80.2M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 68900K .......... .......... .......... .......... .......... 74% 125M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 68950K .......... .......... .......... .......... .......... 74% 61.3M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69000K .......... .......... .......... .......... .......... 74% 82.1M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69050K .......... .......... .......... .......... .......... 74% 40.0M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69100K .......... .......... .......... .......... .......... 74% 122M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69150K .......... .......... .......... .......... .......... 74% 56.6M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69200K .......... .......... .......... .......... .......... 74% 107M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69250K .......... .......... .......... .......... .......... 74% 53.2M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69300K .......... .......... .......... .......... .......... 74% 34.4M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69350K .......... .......... .......... .......... .......... 74% 118M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69400K .......... .......... .......... .......... .......... 74% 49.7M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69450K .......... .......... .......... .......... .......... 74% 82.4M 0s 2024-03-04T18:07:21.819 INFO:teuthology.orchestra.run.smithi047.stderr: 69500K .......... .......... .......... .......... .......... 74% 45.8M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 69550K .......... .......... .......... .......... .......... 74% 112M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 69600K .......... .......... .......... .......... .......... 74% 50.1M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 69650K .......... .......... .......... .......... .......... 75% 47.9M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 69700K .......... .......... .......... .......... .......... 75% 114M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 69750K .......... .......... .......... .......... .......... 75% 50.4M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 69800K .......... .......... .......... .......... .......... 75% 73.8M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 69850K .......... .......... .......... .......... .......... 75% 82.7M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 69900K .......... .......... .......... .......... .......... 75% 126M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 69950K .......... .......... .......... .......... .......... 75% 69.5M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 70000K .......... .......... .......... .......... .......... 75% 94.6M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 70050K .......... .......... .......... .......... .......... 75% 111M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 70100K .......... .......... .......... .......... .......... 75% 59.6M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 70150K .......... .......... .......... .......... .......... 75% 107M 0s 2024-03-04T18:07:21.820 INFO:teuthology.orchestra.run.smithi047.stderr: 70200K .......... .......... .......... .......... .......... 75% 65.5M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70250K .......... .......... .......... .......... .......... 75% 123M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70300K .......... .......... .......... .......... .......... 75% 44.2M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70350K .......... .......... .......... .......... .......... 75% 96.8M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70400K .......... .......... .......... .......... .......... 75% 41.1M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70450K .......... .......... .......... .......... .......... 75% 81.0M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70500K .......... .......... .......... .......... .......... 75% 114M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70550K .......... .......... .......... .......... .......... 75% 48.7M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70600K .......... .......... .......... .......... .......... 76% 41.6M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70650K .......... .......... .......... .......... .......... 76% 78.3M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70700K .......... .......... .......... .......... .......... 76% 51.5M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70750K .......... .......... .......... .......... .......... 76% 107M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70800K .......... .......... .......... .......... .......... 76% 38.9M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70850K .......... .......... .......... .......... .......... 76% 76.7M 0s 2024-03-04T18:07:21.821 INFO:teuthology.orchestra.run.smithi047.stderr: 70900K .......... .......... .......... .......... .......... 76% 76.2M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 70950K .......... .......... .......... .......... .......... 76% 41.3M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71000K .......... .......... .......... .......... .......... 76% 109M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71050K .......... .......... .......... .......... .......... 76% 40.3M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71100K .......... .......... .......... .......... .......... 76% 51.1M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71150K .......... .......... .......... .......... .......... 76% 53.1M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71200K .......... .......... .......... .......... .......... 76% 41.8M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71250K .......... .......... .......... .......... .......... 76% 116M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71300K .......... .......... .......... .......... .......... 76% 57.2M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71350K .......... .......... .......... .......... .......... 76% 59.6M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71400K .......... .......... .......... .......... .......... 76% 42.6M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71450K .......... .......... .......... .......... .......... 76% 75.4M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71500K .......... .......... .......... .......... .......... 77% 63.5M 0s 2024-03-04T18:07:21.822 INFO:teuthology.orchestra.run.smithi047.stderr: 71550K .......... .......... .......... .......... .......... 77% 85.1M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 71600K .......... .......... .......... .......... .......... 77% 43.6M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 71650K .......... .......... .......... .......... .......... 77% 63.5M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 71700K .......... .......... .......... .......... .......... 77% 111M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 71750K .......... .......... .......... .......... .......... 77% 59.4M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 71800K .......... .......... .......... .......... .......... 77% 39.7M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 71850K .......... .......... .......... .......... .......... 77% 86.7M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 71900K .......... .......... .......... .......... .......... 77% 71.7M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 71950K .......... .......... .......... .......... .......... 77% 78.4M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 72000K .......... .......... .......... .......... .......... 77% 75.1M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 72050K .......... .......... .......... .......... .......... 77% 72.3M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 72100K .......... .......... .......... .......... .......... 77% 36.7M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 72150K .......... .......... .......... .......... .......... 77% 112M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 72200K .......... .......... .......... .......... .......... 77% 46.5M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 72250K .......... .......... .......... .......... .......... 77% 69.5M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 72300K .......... .......... .......... .......... .......... 77% 65.0M 0s 2024-03-04T18:07:21.823 INFO:teuthology.orchestra.run.smithi047.stderr: 72350K .......... .......... .......... .......... .......... 77% 120M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72400K .......... .......... .......... .......... .......... 77% 39.6M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72450K .......... .......... .......... .......... .......... 78% 45.2M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72500K .......... .......... .......... .......... .......... 78% 61.1M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72550K .......... .......... .......... .......... .......... 78% 62.3M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72600K .......... .......... .......... .......... .......... 78% 105M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72650K .......... .......... .......... .......... .......... 78% 91.3M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72700K .......... .......... .......... .......... .......... 78% 71.9M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72750K .......... .......... .......... .......... .......... 78% 69.0M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72800K .......... .......... .......... .......... .......... 78% 80.5M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72850K .......... .......... .......... .......... .......... 78% 123M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72900K .......... .......... .......... .......... .......... 78% 66.9M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 72950K .......... .......... .......... .......... .......... 78% 93.0M 0s 2024-03-04T18:07:21.824 INFO:teuthology.orchestra.run.smithi047.stderr: 73000K .......... .......... .......... .......... .......... 78% 43.4M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73050K .......... .......... .......... .......... .......... 78% 125M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73100K .......... .......... .......... .......... .......... 78% 103M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73150K .......... .......... .......... .......... .......... 78% 48.1M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73200K .......... .......... .......... .......... .......... 78% 88.8M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73250K .......... .......... .......... .......... .......... 78% 43.9M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73300K .......... .......... .......... .......... .......... 78% 117M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73350K .......... .......... .......... .......... .......... 79% 93.3M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73400K .......... .......... .......... .......... .......... 79% 49.1M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73450K .......... .......... .......... .......... .......... 79% 79.4M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73500K .......... .......... .......... .......... .......... 79% 112M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73550K .......... .......... .......... .......... .......... 79% 46.4M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73600K .......... .......... .......... .......... .......... 79% 38.2M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73650K .......... .......... .......... .......... .......... 79% 91.9M 0s 2024-03-04T18:07:21.825 INFO:teuthology.orchestra.run.smithi047.stderr: 73700K .......... .......... .......... .......... .......... 79% 70.8M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 73750K .......... .......... .......... .......... .......... 79% 103M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 73800K .......... .......... .......... .......... .......... 79% 43.1M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 73850K .......... .......... .......... .......... .......... 79% 46.5M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 73900K .......... .......... .......... .......... .......... 79% 33.9M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 73950K .......... .......... .......... .......... .......... 79% 97.9M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 74000K .......... .......... .......... .......... .......... 79% 89.0M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 74050K .......... .......... .......... .......... .......... 79% 121M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 74100K .......... .......... .......... .......... .......... 79% 35.9M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 74150K .......... .......... .......... .......... .......... 79% 52.7M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 74200K .......... .......... .......... .......... .......... 79% 50.2M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 74250K .......... .......... .......... .......... .......... 79% 117M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 74300K .......... .......... .......... .......... .......... 80% 37.4M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 74350K .......... .......... .......... .......... .......... 80% 69.9M 0s 2024-03-04T18:07:21.826 INFO:teuthology.orchestra.run.smithi047.stderr: 74400K .......... .......... .......... .......... .......... 80% 52.4M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74450K .......... .......... .......... .......... .......... 80% 38.5M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74500K .......... .......... .......... .......... .......... 80% 111M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74550K .......... .......... .......... .......... .......... 80% 72.8M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74600K .......... .......... .......... .......... .......... 80% 50.7M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74650K .......... .......... .......... .......... .......... 80% 66.1M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74700K .......... .......... .......... .......... .......... 80% 115M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74750K .......... .......... .......... .......... .......... 80% 56.6M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74800K .......... .......... .......... .......... .......... 80% 60.4M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74850K .......... .......... .......... .......... .......... 80% 49.7M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74900K .......... .......... .......... .......... .......... 80% 61.6M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 74950K .......... .......... .......... .......... .......... 80% 89.8M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 75000K .......... .......... .......... .......... .......... 80% 48.7M 0s 2024-03-04T18:07:21.827 INFO:teuthology.orchestra.run.smithi047.stderr: 75050K .......... .......... .......... .......... .......... 80% 79.9M 0s 2024-03-04T18:07:21.828 INFO:teuthology.orchestra.run.smithi047.stderr: 75100K .......... .......... .......... .......... .......... 80% 52.5M 0s 2024-03-04T18:07:21.828 INFO:teuthology.orchestra.run.smithi047.stderr: 75150K .......... .......... .......... .......... .......... 80% 50.2M 0s 2024-03-04T18:07:21.828 INFO:teuthology.orchestra.run.smithi047.stderr: 75200K .......... .......... .......... .......... .......... 81% 100M 0s 2024-03-04T18:07:21.828 INFO:teuthology.orchestra.run.smithi047.stderr: 75250K .......... .......... .......... .......... .......... 81% 48.7M 0s 2024-03-04T18:07:21.828 INFO:teuthology.orchestra.run.smithi047.stderr: 75300K .......... .......... .......... .......... .......... 81% 70.0M 0s 2024-03-04T18:07:21.828 INFO:teuthology.orchestra.run.smithi047.stderr: 75350K .......... .......... .......... .......... .......... 81% 56.2M 0s 2024-03-04T18:07:21.828 INFO:teuthology.orchestra.run.smithi047.stderr: 75400K .......... .......... .......... .......... .......... 81% 109M 0s 2024-03-04T18:07:21.828 INFO:teuthology.orchestra.run.smithi047.stderr: 75450K .......... .......... .......... .......... .......... 81% 96.1M 0s 2024-03-04T18:07:21.828 INFO:teuthology.orchestra.run.smithi047.stderr: 75500K .......... .......... .......... .......... .......... 81% 47.3M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 75550K .......... .......... .......... .......... .......... 81% 81.4M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 75600K .......... .......... .......... .......... .......... 81% 66.6M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 75650K .......... .......... .......... .......... .......... 81% 111M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 75700K .......... .......... .......... .......... .......... 81% 109M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 75750K .......... .......... .......... .......... .......... 81% 46.0M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 75800K .......... .......... .......... .......... .......... 81% 96.1M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 75850K .......... .......... .......... .......... .......... 81% 121M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 75900K .......... .......... .......... .......... .......... 81% 42.5M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 75950K .......... .......... .......... .......... .......... 81% 110M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 76000K .......... .......... .......... .......... .......... 81% 41.8M 0s 2024-03-04T18:07:21.831 INFO:teuthology.orchestra.run.smithi047.stderr: 76050K .......... .......... .......... .......... .......... 81% 94.4M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76100K .......... .......... .......... .......... .......... 81% 95.0M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76150K .......... .......... .......... .......... .......... 82% 55.0M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76200K .......... .......... .......... .......... .......... 82% 72.7M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76250K .......... .......... .......... .......... .......... 82% 54.5M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76300K .......... .......... .......... .......... .......... 82% 64.4M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76350K .......... .......... .......... .......... .......... 82% 121M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76400K .......... .......... .......... .......... .......... 82% 49.0M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76450K .......... .......... .......... .......... .......... 82% 87.0M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76500K .......... .......... .......... .......... .......... 82% 50.9M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76550K .......... .......... .......... .......... .......... 82% 108M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76600K .......... .......... .......... .......... .......... 82% 34.2M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76650K .......... .......... .......... .......... .......... 82% 98.9M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76700K .......... .......... .......... .......... .......... 82% 92.2M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76750K .......... .......... .......... .......... .......... 82% 32.9M 0s 2024-03-04T18:07:21.832 INFO:teuthology.orchestra.run.smithi047.stderr: 76800K .......... .......... .......... .......... .......... 82% 64.1M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 76850K .......... .......... .......... .......... .......... 82% 112M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 76900K .......... .......... .......... .......... .......... 82% 41.1M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 76950K .......... .......... .......... .......... .......... 82% 58.9M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77000K .......... .......... .......... .......... .......... 82% 47.9M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77050K .......... .......... .......... .......... .......... 82% 76.3M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77100K .......... .......... .......... .......... .......... 83% 108M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77150K .......... .......... .......... .......... .......... 83% 37.0M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77200K .......... .......... .......... .......... .......... 83% 52.2M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77250K .......... .......... .......... .......... .......... 83% 59.6M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77300K .......... .......... .......... .......... .......... 83% 116M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77350K .......... .......... .......... .......... .......... 83% 43.5M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77400K .......... .......... .......... .......... .......... 83% 68.3M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77450K .......... .......... .......... .......... .......... 83% 64.3M 0s 2024-03-04T18:07:21.833 INFO:teuthology.orchestra.run.smithi047.stderr: 77500K .......... .......... .......... .......... .......... 83% 63.9M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 77550K .......... .......... .......... .......... .......... 83% 116M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 77600K .......... .......... .......... .......... .......... 83% 43.0M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 77650K .......... .......... .......... .......... .......... 83% 73.2M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 77700K .......... .......... .......... .......... .......... 83% 53.1M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 77750K .......... .......... .......... .......... .......... 83% 121M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 77800K .......... .......... .......... .......... .......... 83% 51.8M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 77850K .......... .......... .......... .......... .......... 83% 42.2M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 77900K .......... .......... .......... .......... .......... 83% 93.8M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 77950K .......... .......... .......... .......... .......... 83% 102M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 78000K .......... .......... .......... .......... .......... 84% 49.5M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 78050K .......... .......... .......... .......... .......... 84% 45.3M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 78100K .......... .......... .......... .......... .......... 84% 65.6M 0s 2024-03-04T18:07:21.834 INFO:teuthology.orchestra.run.smithi047.stderr: 78150K .......... .......... .......... .......... .......... 84% 62.5M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78200K .......... .......... .......... .......... .......... 84% 104M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78250K .......... .......... .......... .......... .......... 84% 45.6M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78300K .......... .......... .......... .......... .......... 84% 101M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78350K .......... .......... .......... .......... .......... 84% 61.8M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78400K .......... .......... .......... .......... .......... 84% 90.1M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78450K .......... .......... .......... .......... .......... 84% 120M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78500K .......... .......... .......... .......... .......... 84% 41.9M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78550K .......... .......... .......... .......... .......... 84% 120M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78600K .......... .......... .......... .......... .......... 84% 50.4M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78650K .......... .......... .......... .......... .......... 84% 104M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78700K .......... .......... .......... .......... .......... 84% 122M 0s 2024-03-04T18:07:21.835 INFO:teuthology.orchestra.run.smithi047.stderr: 78750K .......... .......... .......... .......... .......... 84% 83.3M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 78800K .......... .......... .......... .......... .......... 84% 36.4M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 78850K .......... .......... .......... .......... .......... 84% 122M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 78900K .......... .......... .......... .......... .......... 84% 43.4M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 78950K .......... .......... .......... .......... .......... 85% 115M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79000K .......... .......... .......... .......... .......... 85% 89.6M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79050K .......... .......... .......... .......... .......... 85% 56.8M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79100K .......... .......... .......... .......... .......... 85% 79.1M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79150K .......... .......... .......... .......... .......... 85% 122M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79200K .......... .......... .......... .......... .......... 85% 42.5M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79250K .......... .......... .......... .......... .......... 85% 56.1M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79300K .......... .......... .......... .......... .......... 85% 64.1M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79350K .......... .......... .......... .......... .......... 85% 61.6M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79400K .......... .......... .......... .......... .......... 85% 70.5M 0s 2024-03-04T18:07:21.836 INFO:teuthology.orchestra.run.smithi047.stderr: 79450K .......... .......... .......... .......... .......... 85% 94.4M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79500K .......... .......... .......... .......... .......... 85% 97.0M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79550K .......... .......... .......... .......... .......... 85% 41.3M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79600K .......... .......... .......... .......... .......... 85% 49.2M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79650K .......... .......... .......... .......... .......... 85% 100M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79700K .......... .......... .......... .......... .......... 85% 39.2M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79750K .......... .......... .......... .......... .......... 85% 119M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79800K .......... .......... .......... .......... .......... 85% 32.9M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79850K .......... .......... .......... .......... .......... 86% 122M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79900K .......... .......... .......... .......... .......... 86% 49.5M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 79950K .......... .......... .......... .......... .......... 86% 53.2M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 80000K .......... .......... .......... .......... .......... 86% 52.6M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 80050K .......... .......... .......... .......... .......... 86% 49.8M 0s 2024-03-04T18:07:21.837 INFO:teuthology.orchestra.run.smithi047.stderr: 80100K .......... .......... .......... .......... .......... 86% 108M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80150K .......... .......... .......... .......... .......... 86% 97.4M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80200K .......... .......... .......... .......... .......... 86% 40.0M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80250K .......... .......... .......... .......... .......... 86% 79.3M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80300K .......... .......... .......... .......... .......... 86% 121M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80350K .......... .......... .......... .......... .......... 86% 59.3M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80400K .......... .......... .......... .......... .......... 86% 30.9M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80450K .......... .......... .......... .......... .......... 86% 61.9M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80500K .......... .......... .......... .......... .......... 86% 76.4M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80550K .......... .......... .......... .......... .......... 86% 101M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80600K .......... .......... .......... .......... .......... 86% 72.4M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80650K .......... .......... .......... .......... .......... 86% 41.0M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80700K .......... .......... .......... .......... .......... 86% 95.0M 0s 2024-03-04T18:07:21.838 INFO:teuthology.orchestra.run.smithi047.stderr: 80750K .......... .......... .......... .......... .......... 86% 105M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 80800K .......... .......... .......... .......... .......... 87% 52.0M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 80850K .......... .......... .......... .......... .......... 87% 41.4M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 80900K .......... .......... .......... .......... .......... 87% 68.9M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 80950K .......... .......... .......... .......... .......... 87% 59.8M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81000K .......... .......... .......... .......... .......... 87% 92.8M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81050K .......... .......... .......... .......... .......... 87% 48.6M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81100K .......... .......... .......... .......... .......... 87% 69.3M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81150K .......... .......... .......... .......... .......... 87% 81.5M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81200K .......... .......... .......... .......... .......... 87% 46.7M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81250K .......... .......... .......... .......... .......... 87% 111M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81300K .......... .......... .......... .......... .......... 87% 76.2M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81350K .......... .......... .......... .......... .......... 87% 52.1M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81400K .......... .......... .......... .......... .......... 87% 104M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81450K .......... .......... .......... .......... .......... 87% 114M 0s 2024-03-04T18:07:21.839 INFO:teuthology.orchestra.run.smithi047.stderr: 81500K .......... .......... .......... .......... .......... 87% 49.9M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 81550K .......... .......... .......... .......... .......... 87% 63.6M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 81600K .......... .......... .......... .......... .......... 87% 86.8M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 81650K .......... .......... .......... .......... .......... 87% 48.1M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 81700K .......... .......... .......... .......... .......... 87% 105M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 81750K .......... .......... .......... .......... .......... 88% 104M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 81800K .......... .......... .......... .......... .......... 88% 55.5M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 81850K .......... .......... .......... .......... .......... 88% 59.9M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 81900K .......... .......... .......... .......... .......... 88% 49.1M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 81950K .......... .......... .......... .......... .......... 88% 118M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 82000K .......... .......... .......... .......... .......... 88% 58.3M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 82050K .......... .......... .......... .......... .......... 88% 66.0M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 82100K .......... .......... .......... .......... .......... 88% 47.8M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 82150K .......... .......... .......... .......... .......... 88% 77.4M 0s 2024-03-04T18:07:21.840 INFO:teuthology.orchestra.run.smithi047.stderr: 82200K .......... .......... .......... .......... .......... 88% 74.0M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82250K .......... .......... .......... .......... .......... 88% 79.5M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82300K .......... .......... .......... .......... .......... 88% 48.5M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82350K .......... .......... .......... .......... .......... 88% 66.3M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82400K .......... .......... .......... .......... .......... 88% 109M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82450K .......... .......... .......... .......... .......... 88% 68.6M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82500K .......... .......... .......... .......... .......... 88% 51.1M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82550K .......... .......... .......... .......... .......... 88% 104M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82600K .......... .......... .......... .......... .......... 88% 51.0M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82650K .......... .......... .......... .......... .......... 89% 35.1M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82700K .......... .......... .......... .......... .......... 89% 121M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82750K .......... .......... .......... .......... .......... 89% 58.2M 0s 2024-03-04T18:07:21.841 INFO:teuthology.orchestra.run.smithi047.stderr: 82800K .......... .......... .......... .......... .......... 89% 76.1M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 82850K .......... .......... .......... .......... .......... 89% 37.8M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 82900K .......... .......... .......... .......... .......... 89% 117M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 82950K .......... .......... .......... .......... .......... 89% 74.0M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83000K .......... .......... .......... .......... .......... 89% 49.3M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83050K .......... .......... .......... .......... .......... 89% 50.2M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83100K .......... .......... .......... .......... .......... 89% 121M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83150K .......... .......... .......... .......... .......... 89% 123M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83200K .......... .......... .......... .......... .......... 89% 38.2M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83250K .......... .......... .......... .......... .......... 89% 63.6M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83300K .......... .......... .......... .......... .......... 89% 75.5M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83350K .......... .......... .......... .......... .......... 89% 57.8M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83400K .......... .......... .......... .......... .......... 89% 109M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83450K .......... .......... .......... .......... .......... 89% 42.8M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83500K .......... .......... .......... .......... .......... 89% 96.1M 0s 2024-03-04T18:07:21.842 INFO:teuthology.orchestra.run.smithi047.stderr: 83550K .......... .......... .......... .......... .......... 89% 50.3M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 83600K .......... .......... .......... .......... .......... 90% 121M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 83650K .......... .......... .......... .......... .......... 90% 41.7M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 83700K .......... .......... .......... .......... .......... 90% 90.0M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 83750K .......... .......... .......... .......... .......... 90% 56.7M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 83800K .......... .......... .......... .......... .......... 90% 48.9M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 83850K .......... .......... .......... .......... .......... 90% 121M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 83900K .......... .......... .......... .......... .......... 90% 72.9M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 83950K .......... .......... .......... .......... .......... 90% 58.7M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 84000K .......... .......... .......... .......... .......... 90% 69.3M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 84050K .......... .......... .......... .......... .......... 90% 122M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 84100K .......... .......... .......... .......... .......... 90% 63.4M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 84150K .......... .......... .......... .......... .......... 90% 64.1M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 84200K .......... .......... .......... .......... .......... 90% 95.5M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 84250K .......... .......... .......... .......... .......... 90% 43.1M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 84300K .......... .......... .......... .......... .......... 90% 93.4M 0s 2024-03-04T18:07:21.843 INFO:teuthology.orchestra.run.smithi047.stderr: 84350K .......... .......... .......... .......... .......... 90% 72.6M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84400K .......... .......... .......... .......... .......... 90% 48.2M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84450K .......... .......... .......... .......... .......... 90% 94.3M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84500K .......... .......... .......... .......... .......... 91% 52.6M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84550K .......... .......... .......... .......... .......... 91% 65.1M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84600K .......... .......... .......... .......... .......... 91% 73.8M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84650K .......... .......... .......... .......... .......... 91% 97.8M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84700K .......... .......... .......... .......... .......... 91% 58.8M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84750K .......... .......... .......... .......... .......... 91% 61.5M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84800K .......... .......... .......... .......... .......... 91% 74.5M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84850K .......... .......... .......... .......... .......... 91% 83.0M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84900K .......... .......... .......... .......... .......... 91% 86.9M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 84950K .......... .......... .......... .......... .......... 91% 53.6M 0s 2024-03-04T18:07:21.844 INFO:teuthology.orchestra.run.smithi047.stderr: 85000K .......... .......... .......... .......... .......... 91% 119M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85050K .......... .......... .......... .......... .......... 91% 52.5M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85100K .......... .......... .......... .......... .......... 91% 44.3M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85150K .......... .......... .......... .......... .......... 91% 74.4M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85200K .......... .......... .......... .......... .......... 91% 66.7M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85250K .......... .......... .......... .......... .......... 91% 57.0M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85300K .......... .......... .......... .......... .......... 91% 142M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85350K .......... .......... .......... .......... .......... 91% 62.5M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85400K .......... .......... .......... .......... .......... 91% 57.3M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85450K .......... .......... .......... .......... .......... 92% 49.9M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85500K .......... .......... .......... .......... .......... 92% 98.9M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85550K .......... .......... .......... .......... .......... 92% 89.6M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85600K .......... .......... .......... .......... .......... 92% 62.2M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85650K .......... .......... .......... .......... .......... 92% 72.7M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85700K .......... .......... .......... .......... .......... 92% 125M 0s 2024-03-04T18:07:21.845 INFO:teuthology.orchestra.run.smithi047.stderr: 85750K .......... .......... .......... .......... .......... 92% 33.6M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 85800K .......... .......... .......... .......... .......... 92% 66.9M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 85850K .......... .......... .......... .......... .......... 92% 51.7M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 85900K .......... .......... .......... .......... .......... 92% 74.0M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 85950K .......... .......... .......... .......... .......... 92% 123M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86000K .......... .......... .......... .......... .......... 92% 32.2M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86050K .......... .......... .......... .......... .......... 92% 59.7M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86100K .......... .......... .......... .......... .......... 92% 50.7M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86150K .......... .......... .......... .......... .......... 92% 151M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86200K .......... .......... .......... .......... .......... 92% 134M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86250K .......... .......... .......... .......... .......... 92% 61.0M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86300K .......... .......... .......... .......... .......... 92% 68.6M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86350K .......... .......... .......... .......... .......... 93% 72.5M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86400K .......... .......... .......... .......... .......... 93% 111M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86450K .......... .......... .......... .......... .......... 93% 43.8M 0s 2024-03-04T18:07:21.846 INFO:teuthology.orchestra.run.smithi047.stderr: 86500K .......... .......... .......... .......... .......... 93% 71.9M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 86550K .......... .......... .......... .......... .......... 93% 47.3M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 86600K .......... .......... .......... .......... .......... 93% 112M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 86650K .......... .......... .......... .......... .......... 93% 65.0M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 86700K .......... .......... .......... .......... .......... 93% 96.6M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 86750K .......... .......... .......... .......... .......... 93% 50.6M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 86800K .......... .......... .......... .......... .......... 93% 79.2M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 86850K .......... .......... .......... .......... .......... 93% 74.9M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 86900K .......... .......... .......... .......... .......... 93% 74.6M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 86950K .......... .......... .......... .......... .......... 93% 107M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 87000K .......... .......... .......... .......... .......... 93% 45.1M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 87050K .......... .......... .......... .......... .......... 93% 88.0M 0s 2024-03-04T18:07:21.847 INFO:teuthology.orchestra.run.smithi047.stderr: 87100K .......... .......... .......... .......... .......... 93% 75.3M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87150K .......... .......... .......... .......... .......... 93% 79.8M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87200K .......... .......... .......... .......... .......... 93% 53.5M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87250K .......... .......... .......... .......... .......... 93% 96.3M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87300K .......... .......... .......... .......... .......... 94% 102M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87350K .......... .......... .......... .......... .......... 94% 72.6M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87400K .......... .......... .......... .......... .......... 94% 61.9M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87450K .......... .......... .......... .......... .......... 94% 117M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87500K .......... .......... .......... .......... .......... 94% 46.4M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87550K .......... .......... .......... .......... .......... 94% 113M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87600K .......... .......... .......... .......... .......... 94% 38.5M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87650K .......... .......... .......... .......... .......... 94% 103M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87700K .......... .......... .......... .......... .......... 94% 59.5M 0s 2024-03-04T18:07:21.848 INFO:teuthology.orchestra.run.smithi047.stderr: 87750K .......... .......... .......... .......... .......... 94% 119M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 87800K .......... .......... .......... .......... .......... 94% 68.1M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 87850K .......... .......... .......... .......... .......... 94% 49.9M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 87900K .......... .......... .......... .......... .......... 94% 54.2M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 87950K .......... .......... .......... .......... .......... 94% 111M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88000K .......... .......... .......... .......... .......... 94% 49.6M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88050K .......... .......... .......... .......... .......... 94% 96.9M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88100K .......... .......... .......... .......... .......... 94% 52.9M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88150K .......... .......... .......... .......... .......... 94% 41.6M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88200K .......... .......... .......... .......... .......... 94% 64.5M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88250K .......... .......... .......... .......... .......... 95% 82.5M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88300K .......... .......... .......... .......... .......... 95% 117M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88350K .......... .......... .......... .......... .......... 95% 104M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88400K .......... .......... .......... .......... .......... 95% 49.7M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88450K .......... .......... .......... .......... .......... 95% 114M 0s 2024-03-04T18:07:21.849 INFO:teuthology.orchestra.run.smithi047.stderr: 88500K .......... .......... .......... .......... .......... 95% 37.6M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 88550K .......... .......... .......... .......... .......... 95% 51.7M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 88600K .......... .......... .......... .......... .......... 95% 102M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 88650K .......... .......... .......... .......... .......... 95% 45.6M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 88700K .......... .......... .......... .......... .......... 95% 100M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 88750K .......... .......... .......... .......... .......... 95% 117M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 88800K .......... .......... .......... .......... .......... 95% 31.0M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 88850K .......... .......... .......... .......... .......... 95% 112M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 88900K .......... .......... .......... .......... .......... 95% 82.5M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 88950K .......... .......... .......... .......... .......... 95% 45.4M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 89000K .......... .......... .......... .......... .......... 95% 116M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 89050K .......... .......... .......... .......... .......... 95% 40.8M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 89100K .......... .......... .......... .......... .......... 95% 102M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 89150K .......... .......... .......... .......... .......... 96% 47.8M 0s 2024-03-04T18:07:21.850 INFO:teuthology.orchestra.run.smithi047.stderr: 89200K .......... .......... .......... .......... .......... 96% 118M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89250K .......... .......... .......... .......... .......... 96% 112M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89300K .......... .......... .......... .......... .......... 96% 48.1M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89350K .......... .......... .......... .......... .......... 96% 31.0M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89400K .......... .......... .......... .......... .......... 96% 41.3M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89450K .......... .......... .......... .......... .......... 96% 120M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89500K .......... .......... .......... .......... .......... 96% 21.8M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89550K .......... .......... .......... .......... .......... 96% 31.5M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89600K .......... .......... .......... .......... .......... 96% 72.3M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89650K .......... .......... .......... .......... .......... 96% 108M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89700K .......... .......... .......... .......... .......... 96% 97.9M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89750K .......... .......... .......... .......... .......... 96% 60.7M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89800K .......... .......... .......... .......... .......... 96% 50.7M 0s 2024-03-04T18:07:21.851 INFO:teuthology.orchestra.run.smithi047.stderr: 89850K .......... .......... .......... .......... .......... 96% 84.9M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 89900K .......... .......... .......... .......... .......... 96% 118M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 89950K .......... .......... .......... .......... .......... 96% 22.9M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90000K .......... .......... .......... .......... .......... 96% 80.7M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90050K .......... .......... .......... .......... .......... 96% 81.2M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90100K .......... .......... .......... .......... .......... 97% 121M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90150K .......... .......... .......... .......... .......... 97% 45.2M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90200K .......... .......... .......... .......... .......... 97% 63.5M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90250K .......... .......... .......... .......... .......... 97% 85.3M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90300K .......... .......... .......... .......... .......... 97% 48.4M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90350K .......... .......... .......... .......... .......... 97% 109M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90400K .......... .......... .......... .......... .......... 97% 106M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90450K .......... .......... .......... .......... .......... 97% 67.3M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90500K .......... .......... .......... .......... .......... 97% 56.5M 0s 2024-03-04T18:07:21.852 INFO:teuthology.orchestra.run.smithi047.stderr: 90550K .......... .......... .......... .......... .......... 97% 81.8M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 90600K .......... .......... .......... .......... .......... 97% 106M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 90650K .......... .......... .......... .......... .......... 97% 41.2M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 90700K .......... .......... .......... .......... .......... 97% 56.9M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 90750K .......... .......... .......... .......... .......... 97% 60.9M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 90800K .......... .......... .......... .......... .......... 97% 114M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 90850K .......... .......... .......... .......... .......... 97% 107M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 90900K .......... .......... .......... .......... .......... 97% 48.2M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 90950K .......... .......... .......... .......... .......... 97% 36.1M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 91000K .......... .......... .......... .......... .......... 98% 51.2M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 91050K .......... .......... .......... .......... .......... 98% 110M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 91100K .......... .......... .......... .......... .......... 98% 102M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 91150K .......... .......... .......... .......... .......... 98% 83.7M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 91200K .......... .......... .......... .......... .......... 98% 46.2M 0s 2024-03-04T18:07:21.853 INFO:teuthology.orchestra.run.smithi047.stderr: 91250K .......... .......... .......... .......... .......... 98% 115M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91300K .......... .......... .......... .......... .......... 98% 70.2M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91350K .......... .......... .......... .......... .......... 98% 44.9M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91400K .......... .......... .......... .......... .......... 98% 86.5M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91450K .......... .......... .......... .......... .......... 98% 41.2M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91500K .......... .......... .......... .......... .......... 98% 64.5M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91550K .......... .......... .......... .......... .......... 98% 83.6M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91600K .......... .......... .......... .......... .......... 98% 47.3M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91650K .......... .......... .......... .......... .......... 98% 138M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91700K .......... .......... .......... .......... .......... 98% 54.2M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91750K .......... .......... .......... .......... .......... 98% 66.7M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91800K .......... .......... .......... .......... .......... 98% 127M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91850K .......... .......... .......... .......... .......... 98% 41.5M 0s 2024-03-04T18:07:21.854 INFO:teuthology.orchestra.run.smithi047.stderr: 91900K .......... .......... .......... .......... .......... 98% 78.4M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 91950K .......... .......... .......... .......... .......... 99% 113M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92000K .......... .......... .......... .......... .......... 99% 47.5M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92050K .......... .......... .......... .......... .......... 99% 110M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92100K .......... .......... .......... .......... .......... 99% 49.2M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92150K .......... .......... .......... .......... .......... 99% 52.1M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92200K .......... .......... .......... .......... .......... 99% 103M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92250K .......... .......... .......... .......... .......... 99% 115M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92300K .......... .......... .......... .......... .......... 99% 119M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92350K .......... .......... .......... .......... .......... 99% 87.4M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92400K .......... .......... .......... .......... .......... 99% 45.2M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92450K .......... .......... .......... .......... .......... 99% 49.5M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92500K .......... .......... .......... .......... .......... 99% 116M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92550K .......... .......... .......... .......... .......... 99% 46.6M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92600K .......... .......... .......... .......... .......... 99% 86.1M 0s 2024-03-04T18:07:21.855 INFO:teuthology.orchestra.run.smithi047.stderr: 92650K .......... .......... .......... .......... .......... 99% 85.6M 0s 2024-03-04T18:07:21.856 INFO:teuthology.orchestra.run.smithi047.stderr: 92700K .......... .......... .......... .......... .......... 99% 114M 0s 2024-03-04T18:07:21.856 INFO:teuthology.orchestra.run.smithi047.stderr: 92750K .......... .......... .......... .......... .......... 99% 45.9M 0s 2024-03-04T18:07:21.856 INFO:teuthology.orchestra.run.smithi047.stderr: 92800K .......... .......... .......... .......... .......... 99% 79.1M 0s 2024-03-04T18:07:21.856 INFO:teuthology.orchestra.run.smithi047.stderr: 92850K .......... .......... .......... .......... .......... 99% 81.9M 0s 2024-03-04T18:07:21.856 INFO:teuthology.orchestra.run.smithi047.stderr: 92900K 100% 134G=1.6s 2024-03-04T18:07:21.856 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-03-04T18:07:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04 18:07:21 (58.5 MB/s) - 'gradle-6.0.1-bin.zip' saved [95129672/95129672] 2024-03-04T18:07:21.856 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-03-04T18:07:21.862 INFO:teuthology.orchestra.run.smithi047.stdout:Archive: gradle-6.0.1-bin.zip 2024-03-04T18:07:21.862 INFO:teuthology.orchestra.run.smithi047.stdout: creating: /opt/gradle/gradle-6.0.1/ 2024-03-04T18:07:21.862 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/getting-started.html 2024-03-04T18:07:21.862 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/LICENSE 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/NOTICE 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: creating: /opt/gradle/gradle-6.0.1/init.d/ 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/init.d/readme.txt 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: creating: /opt/gradle/gradle-6.0.1/bin/ 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle.bat 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: creating: /opt/gradle/gradle-6.0.1/lib/ 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-api-metadata-6.0.1.jar 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-launcher-6.0.1.jar 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-runtime-api-info-6.0.1.jar 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-6.0.1.jar 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-tooling-api-6.0.1.jar 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-wrapper-6.0.1.jar 2024-03-04T18:07:21.863 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-installation-beacon-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-bootstrap-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-file-collections-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-groovy-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-execution-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-core-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-packaging-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-api-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-persistent-cache-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-worker-processes-6.0.1.jar 2024-03-04T18:07:21.864 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-logging-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-jvm-services-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-process-services-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-resources-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-messaging-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-native-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-groovy-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-option-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-cli-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-snapshots-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-files-6.0.1.jar 2024-03-04T18:07:21.865 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-tooling-models-6.0.1.jar 2024-03-04T18:07:22.050 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar 2024-03-04T18:07:22.050 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-hashing-6.0.1.jar 2024-03-04T18:07:22.050 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-pineapple-6.0.1.jar 2024-03-04T18:07:22.050 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-docs-6.0.1.jar 2024-03-04T18:07:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/groovy-all-1.3-2.5.8.jar 2024-03-04T18:07:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jul-to-slf4j-1.7.28.jar 2024-03-04T18:07:22.106 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/log4j-over-slf4j-1.7.28.jar 2024-03-04T18:07:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jcl-over-slf4j-1.7.28.jar 2024-03-04T18:07:22.107 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/slf4j-api-1.7.28.jar 2024-03-04T18:07:22.124 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/guava-27.1-android.jar 2024-03-04T18:07:22.126 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-io-2.6.jar 2024-03-04T18:07:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-lang-2.6.jar 2024-03-04T18:07:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-commons-7.1.jar 2024-03-04T18:07:22.128 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-analysis-7.1.jar 2024-03-04T18:07:22.129 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-tree-7.1.jar 2024-03-04T18:07:22.130 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-7.1.jar 2024-03-04T18:07:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-1.10.7.jar 2024-03-04T18:07:22.145 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk8-1.3.50.jar 2024-03-04T18:07:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/javax.inject-1.jar 2024-03-04T18:07:22.146 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-common-1.3.50.jar 2024-03-04T18:07:22.147 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-1.3.50.jar 2024-03-04T18:07:22.148 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-host-embeddable-1.3.50.jar 2024-03-04T18:07:22.150 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-embeddable-1.3.50.jar 2024-03-04T18:07:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-impl-embeddable-1.3.50.jar 2024-03-04T18:07:22.152 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-sam-with-receiver-compiler-plugin-1.3.50.jar 2024-03-04T18:07:22.168 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlinx-metadata-jvm-0.1.0.jar 2024-03-04T18:07:22.172 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-compress-1.19.jar 2024-03-04T18:07:22.176 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/fastutil-8.3.0-min.jar 2024-03-04T18:07:22.178 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jansi-1.18.jar 2024-03-04T18:07:22.178 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jsr305-3.0.2.jar 2024-03-04T18:07:22.180 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kryo-2.24.0.jar 2024-03-04T18:07:22.181 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-0.18.jar 2024-03-04T18:07:22.181 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/objenesis-2.6.jar 2024-03-04T18:07:22.185 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/trove4j-1.0.20181211.jar 2024-03-04T18:07:22.187 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/xml-apis-1.4.01.jar 2024-03-04T18:07:22.207 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-reflect-1.3.50.jar 2024-03-04T18:07:22.207 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk7-1.3.50.jar 2024-03-04T18:07:22.217 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-1.3.50.jar 2024-03-04T18:07:22.217 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/annotations-13.0.jar 2024-03-04T18:07:22.217 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-launcher-1.10.7.jar 2024-03-04T18:07:22.217 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/minlog-1.2.jar 2024-03-04T18:07:22.217 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-script-runtime-1.3.50.jar 2024-03-04T18:07:22.224 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-daemon-embeddable-1.3.50.jar 2024-03-04T18:07:22.225 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/failureaccess-1.0.1.jar 2024-03-04T18:07:22.227 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-common-1.3.50.jar 2024-03-04T18:07:22.227 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libcpp-0.18.jar 2024-03-04T18:07:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libstdcpp-0.18.jar 2024-03-04T18:07:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libcpp-0.18.jar 2024-03-04T18:07:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libstdcpp-0.18.jar 2024-03-04T18:07:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-0.18.jar 2024-03-04T18:07:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-0.18.jar 2024-03-04T18:07:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-0.18.jar 2024-03-04T18:07:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-osx-amd64-0.18.jar 2024-03-04T18:07:22.228 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-0.18.jar 2024-03-04T18:07:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-min-0.18.jar 2024-03-04T18:07:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-0.18.jar 2024-03-04T18:07:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-min-0.18.jar 2024-03-04T18:07:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-ncurses5-0.18.jar 2024-03-04T18:07:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses5-0.18.jar 2024-03-04T18:07:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses6-0.18.jar 2024-03-04T18:07:22.231 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses5-0.18.jar 2024-03-04T18:07:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses6-0.18.jar 2024-03-04T18:07:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: creating: /opt/gradle/gradle-6.0.1/lib/plugins/ 2024-03-04T18:07:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-antlr-6.0.1.jar 2024-03-04T18:07:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-code-quality-6.0.1.jar 2024-03-04T18:07:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-signing-6.0.1.jar 2024-03-04T18:07:22.232 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-native-6.0.1.jar 2024-03-04T18:07:22.233 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-native-6.0.1.jar 2024-03-04T18:07:22.233 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-native-6.0.1.jar 2024-03-04T18:07:22.236 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-native-6.0.1.jar 2024-03-04T18:07:22.240 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-provider-plugins-6.0.1.jar 2024-03-04T18:07:22.240 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-development-6.0.1.jar 2024-03-04T18:07:22.240 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-maven-6.0.1.jar 2024-03-04T18:07:22.240 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-play-6.0.1.jar 2024-03-04T18:07:22.243 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-6.0.1.jar 2024-03-04T18:07:22.243 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-scala-6.0.1.jar 2024-03-04T18:07:22.243 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ear-6.0.1.jar 2024-03-04T18:07:22.245 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-play-6.0.1.jar 2024-03-04T18:07:22.245 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-javascript-6.0.1.jar 2024-03-04T18:07:22.246 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ivy-6.0.1.jar 2024-03-04T18:07:22.247 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-jacoco-6.0.1.jar 2024-03-04T18:07:22.249 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-init-6.0.1.jar 2024-03-04T18:07:22.249 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-tooling-builders-6.0.1.jar 2024-03-04T18:07:22.252 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugins-6.0.1.jar 2024-03-04T18:07:22.252 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-test-kit-6.0.1.jar 2024-03-04T18:07:22.253 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-groovy-6.0.1.jar 2024-03-04T18:07:22.253 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-scala-6.0.1.jar 2024-03-04T18:07:22.253 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-junit-platform-6.0.1.jar 2024-03-04T18:07:22.254 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-api-builders-6.0.1.jar 2024-03-04T18:07:22.254 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-jvm-6.0.1.jar 2024-03-04T18:07:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-java-6.0.1.jar 2024-03-04T18:07:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-jvm-6.0.1.jar 2024-03-04T18:07:22.258 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-jvm-6.0.1.jar 2024-03-04T18:07:22.261 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-native-6.0.1.jar 2024-03-04T18:07:22.264 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-diagnostics-6.0.1.jar 2024-03-04T18:07:22.264 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-publish-6.0.1.jar 2024-03-04T18:07:22.265 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-base-6.0.1.jar 2024-03-04T18:07:22.267 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-base-6.0.1.jar 2024-03-04T18:07:22.267 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-use-6.0.1.jar 2024-03-04T18:07:22.274 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-instant-execution-6.0.1.jar 2024-03-04T18:07:22.274 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-version-control-6.0.1.jar 2024-03-04T18:07:22.289 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-dependency-management-6.0.1.jar 2024-03-04T18:07:22.289 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-composite-builds-6.0.1.jar 2024-03-04T18:07:22.290 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-reporting-6.0.1.jar 2024-03-04T18:07:22.290 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-s3-6.0.1.jar 2024-03-04T18:07:22.290 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-gcs-6.0.1.jar 2024-03-04T18:07:22.290 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-cache-http-6.0.1.jar 2024-03-04T18:07:22.291 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-http-6.0.1.jar 2024-03-04T18:07:22.291 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-sftp-6.0.1.jar 2024-03-04T18:07:22.291 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-profile-6.0.1.jar 2024-03-04T18:07:22.292 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-workers-6.0.1.jar 2024-03-04T18:07:22.311 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/org.eclipse.jgit-5.5.0.201909110433-r.jar 2024-03-04T18:07:22.311 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-3.0.0.jar 2024-03-04T18:07:22.311 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-shared-3.0.0.jar 2024-03-04T18:07:22.312 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-services-storage-v1-rev136-1.25.0.jar 2024-03-04T18:07:22.313 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-client-1.25.0.jar 2024-03-04T18:07:22.314 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/asm-util-7.1.jar 2024-03-04T18:07:22.322 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ivy-2.3.0.jar 2024-03-04T18:07:22.324 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-compat-3.0.4.jar 2024-03-04T18:07:22.329 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-core-3.0.4.jar 2024-03-04T18:07:22.329 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-common-0.8-20100325.jar 2024-03-04T18:07:22.329 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-groovy-0.8-20100325.jar 2024-03-04T18:07:22.329 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-file-3.0.0.jar 2024-03-04T18:07:22.330 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-container-default-1.7.1.jar 2024-03-04T18:07:22.331 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-connector-wagon-1.13.1.jar 2024-03-04T18:07:22.333 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpg-jdk15on-1.63.jar 2024-03-04T18:07:22.341 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/rhino-1.7.10.jar 2024-03-04T18:07:22.343 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gson-2.8.5.jar 2024-03-04T18:07:22.345 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/simple-4.1.21.jar 2024-03-04T18:07:22.345 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jatl-0.2.3.jar 2024-03-04T18:07:22.346 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-plugin-api-3.0.4.jar 2024-03-04T18:07:22.346 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-oauth-client-1.25.0.jar 2024-03-04T18:07:22.346 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-jackson2-1.25.0.jar 2024-03-04T18:07:22.349 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-1.25.0.jar 2024-03-04T18:07:22.353 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/testng-6.3.1.jar 2024-03-04T18:07:22.355 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/snakeyaml-1.17.jar 2024-03-04T18:07:22.355 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/dd-plist-1.21.jar 2024-03-04T18:07:22.358 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-4.12.jar 2024-03-04T18:07:22.360 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bsh-2.0b6.jar 2024-03-04T18:07:22.361 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-launcher-1.3.1.jar 2024-03-04T18:07:22.368 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-s3-1.11.633.jar 2024-03-04T18:07:22.371 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-kms-1.11.633.jar 2024-03-04T18:07:22.378 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-core-1.11.633.jar 2024-03-04T18:07:22.383 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpclient-4.5.10.jar 2024-03-04T18:07:22.386 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcifs-1.3.17.jar 2024-03-04T18:07:22.387 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/nekohtml-1.9.22.jar 2024-03-04T18:07:22.396 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xercesImpl-2.12.0.jar 2024-03-04T18:07:22.398 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jsch-0.1.55.jar 2024-03-04T18:07:22.405 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jaxb-impl-2.3.2.jar 2024-03-04T18:07:22.405 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jmespath-java-1.11.633.jar 2024-03-04T18:07:22.414 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-databind-2.9.10.jar 2024-03-04T18:07:22.416 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-core-2.9.10.jar 2024-03-04T18:07:22.417 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-annotations-2.9.10.jar 2024-03-04T18:07:22.421 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/joda-time-2.10.4.jar 2024-03-04T18:07:22.427 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpkix-jdk15on-1.61.jar 2024-03-04T18:07:22.458 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcprov-jdk15on-1.63.jar 2024-03-04T18:07:22.461 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/commons-codec-1.13.jar 2024-03-04T18:07:22.461 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-aether-provider-3.0.4.jar 2024-03-04T18:07:22.462 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-impl-1.13.1.jar 2024-03-04T18:07:22.463 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-util-1.13.1.jar 2024-03-04T18:07:22.463 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-spi-1.13.1.jar 2024-03-04T18:07:22.463 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-api-1.13.1.jar 2024-03-04T18:07:22.464 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/hamcrest-core-1.3.jar 2024-03-04T18:07:22.466 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpcore-4.4.12.jar 2024-03-04T18:07:22.466 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcommander-1.72.jar 2024-03-04T18:07:22.467 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jzlib-1.1.3.jar 2024-03-04T18:07:22.467 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-artifact-3.0.4.jar 2024-03-04T18:07:22.468 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-builder-3.0.4.jar 2024-03-04T18:07:22.470 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-3.0.4.jar 2024-03-04T18:07:22.470 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-repository-metadata-3.0.4.jar 2024-03-04T18:07:22.471 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-builder-3.0.4.jar 2024-03-04T18:07:22.471 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-3.0.4.jar 2024-03-04T18:07:22.471 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-sec-dispatcher-1.3.jar 2024-03-04T18:07:22.471 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-cipher-1.7.jar 2024-03-04T18:07:22.472 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-classworlds-2.5.1.jar 2024-03-04T18:07:22.472 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-component-annotations-1.5.5.jar 2024-03-04T18:07:22.472 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-interpolation-1.14.jar 2024-03-04T18:07:22.472 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-provider-api-3.0.0.jar 2024-03-04T18:07:22.474 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-utils-3.1.0.jar 2024-03-04T18:07:22.475 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xbean-reflect-3.7.jar 2024-03-04T18:07:22.479 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ion-java-1.0.2.jar 2024-03-04T18:07:22.480 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-engine-1.3.1.jar 2024-03-04T18:07:22.481 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-commons-1.3.1.jar 2024-03-04T18:07:22.481 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/apiguardian-api-1.0.0.jar 2024-03-04T18:07:22.481 INFO:teuthology.orchestra.run.smithi047.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/opentest4j-1.1.1.jar 2024-03-04T18:07:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:export PATH=/opt/gradle/gradle-6.0.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 2024-03-04T18:07:23.000 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:23.000 INFO:teuthology.orchestra.run.smithi047.stdout:Welcome to Gradle 6.0.1! 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout:Here are the highlights of this release: 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - Substantial improvements in dependency management, including 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - Publishing Gradle Module Metadata in addition to pom.xml 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - Advanced control of transitive versions 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - Support for optional features and dependencies 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - Rules to tweak published metadata 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - Support for Java 13 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - Faster incremental Java and Groovy compilation 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - New Zinc compiler for Scala 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - VS2019 support 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: - Support for Gradle Enterprise plugin 3.0 2024-03-04T18:07:23.001 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:For more details see https://docs.gradle.org/6.0.1/release-notes.html 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:------------------------------------------------------------ 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:Gradle 6.0.1 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:------------------------------------------------------------ 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:Build time: 2019-11-18 20:25:01 UTC 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:Revision: fad121066a68c4701acd362daf4287a7c309a0f5 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:Kotlin: 1.3.50 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:Groovy: 2.5.8 2024-03-04T18:07:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 2024-03-04T18:07:23.003 INFO:teuthology.orchestra.run.smithi047.stdout:JVM: 11.0.17 (Ubuntu 11.0.17+8-post-Ubuntu-1ubuntu222.04) 2024-03-04T18:07:23.003 INFO:teuthology.orchestra.run.smithi047.stdout:OS: Linux 5.15.0-97-generic amd64 2024-03-04T18:07:23.003 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:23.019 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2024-03-04T18:07:23.019 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-03-04T18:07:23.019 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi047.front.sepia.ceph.com 2024-03-04T18:07:23.019 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-03-04T18:07:23.020 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi152.front.sepia.ceph.com 2024-03-04T18:07:23.020 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2024-03-04T18:07:23.231 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 'access_secret': None, 'region': 'mexico', 'endpoint': None, 'port': None, 'display_name': None, 'email': None, 'is_secure': None, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data'}} 2024-03-04T18:07:23.231 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2024-03-04T18:07:23.231 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'PGHYMMXPJITOUTKGJVBK', 'access_secret': 'MFCwdw4hIab/pucALqgAArBLuj8b4aBks1GmfdRcNLmojZINX8TJuw==', 'region': 'mexico', 'endpoint': 'smithi047.front.sepia.ceph.com:443', 'port': 443, 'display_name': 'Ms. tester.client.0', 'email': 'tester.client.0_test@test.test', 'is_secure': True, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data', 'user_id': 'tester.client.0', 'host': 'smithi047.front.sepia.ceph.com'} 2024-03-04T18:07:23.231 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2024-03-04T18:07:23.232 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'PGHYMMXPJITOUTKGJVBK', '--secret', 'MFCwdw4hIab/pucALqgAArBLuj8b4aBks1GmfdRcNLmojZINX8TJuw==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2024-03-04T18:07:23.232 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key PGHYMMXPJITOUTKGJVBK --secret MFCwdw4hIab/pucALqgAArBLuj8b4aBks1GmfdRcNLmojZINX8TJuw== --email tester.client.0_test@test.test --cluster ceph 2024-03-04T18:07:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:07:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:07:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.320+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 realm 2024-03-04T18:07:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:07:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:07:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:07:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.324+0000 7f219460f980 20 searching for the correct realm 2024-03-04T18:07:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.332+0000 7f2191926640 0 -- 172.21.15.47:0/3021251704 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f2188045920 msgr2=0x7f218802e400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:07:23.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.340+0000 7f2192928640 0 -- 172.21.15.47:0/3021251704 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x563bb3871860 msgr2=0x563bb3873d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:07:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:07:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:07:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:07:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:07:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:07:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:07:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:07:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:07:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 zone default found 2024-03-04T18:07:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 4 Realm: () 2024-03-04T18:07:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:07:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:07:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:07:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 zonegroup default 2024-03-04T18:07:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.348+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.352+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.352+0000 7f219460f980 20 started sync module instance, tier type = 2024-03-04T18:07:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.352+0000 7f219460f980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:07:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.368+0000 7f219460f980 20 add_watcher() i=0 2024-03-04T18:07:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.368+0000 7f219460f980 20 add_watcher() i=1 2024-03-04T18:07:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.368+0000 7f219460f980 20 add_watcher() i=2 2024-03-04T18:07:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.368+0000 7f219460f980 20 add_watcher() i=3 2024-03-04T18:07:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.368+0000 7f219460f980 20 add_watcher() i=4 2024-03-04T18:07:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.368+0000 7f219460f980 20 add_watcher() i=5 2024-03-04T18:07:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.368+0000 7f219460f980 20 add_watcher() i=6 2024-03-04T18:07:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.372+0000 7f219460f980 20 add_watcher() i=7 2024-03-04T18:07:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.372+0000 7f219460f980 2 all 8 watchers are set, enabling cache 2024-03-04T18:07:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f2048fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:07:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:07:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:07:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 5 note: GC not initialized 2024-03-04T18:07:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f1fc37fe640 20 reqs_thread_entry: start 2024-03-04T18:07:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 init_complete bucket index max shards: 11 2024-03-04T18:07:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 Filter name: none 2024-03-04T18:07:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f1fc17fa640 20 reqs_thread_entry: start 2024-03-04T18:07:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-03-04T18:07:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2024-03-04T18:07:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-03-04T18:07:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2024-03-04T18:07:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 cache get: name=default.rgw.meta+users.keys+PGHYMMXPJITOUTKGJVBK : miss 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 cache put: name=default.rgw.meta+users.keys+PGHYMMXPJITOUTKGJVBK info.flags=0x0 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 adding default.rgw.meta+users.keys+PGHYMMXPJITOUTKGJVBK to cache LRU end 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 cache get: name=default.rgw.meta+users.keys+PGHYMMXPJITOUTKGJVBK : hit (negative entry) 2024-03-04T18:07:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.376+0000 7f219460f980 10 cache get: name=default.rgw.meta+users.keys+PGHYMMXPJITOUTKGJVBK : hit (negative entry) 2024-03-04T18:07:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.380+0000 7f219460f980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-03-04T18:07:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.380+0000 7f219460f980 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-03-04T18:07:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.380+0000 7f219460f980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-03-04T18:07:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.380+0000 7f2075ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595130 cookie 94814415098384 notifier 7356 bl.length()=637 2024-03-04T18:07:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.380+0000 7f2075ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-03-04T18:07:23.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.380+0000 7f2075ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-03-04T18:07:23.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.384+0000 7f219460f980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-03-04T18:07:23.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.384+0000 7f219460f980 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-03-04T18:07:23.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.384+0000 7f219460f980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-03-04T18:07:23.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.384+0000 7f2075ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595120 cookie 94814415088720 notifier 7356 bl.length()=200 2024-03-04T18:07:23.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.384+0000 7f2075ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-03-04T18:07:23.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.384+0000 7f2075ffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-03-04T18:07:23.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.388+0000 7f219460f980 10 cache put: name=default.rgw.meta+users.keys+PGHYMMXPJITOUTKGJVBK info.flags=0x7 2024-03-04T18:07:23.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.388+0000 7f219460f980 10 moving default.rgw.meta+users.keys+PGHYMMXPJITOUTKGJVBK to cache LRU end 2024-03-04T18:07:23.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.388+0000 7f219460f980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:PGHYMMXPJITOUTKGJVBK, ofs0, ns] 2024-03-04T18:07:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.392+0000 7f2075ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595131 cookie 94814415092000 notifier 7356 bl.length()=189 2024-03-04T18:07:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.392+0000 7f2075ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+PGHYMMXPJITOUTKGJVBK info.flags=0x7 2024-03-04T18:07:23.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.392+0000 7f2075ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+PGHYMMXPJITOUTKGJVBK to cache LRU end 2024-03-04T18:07:23.390 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-04T18:07:23.390 INFO:teuthology.orchestra.run.smithi047.stdout: "user_id": "tester.client.0", 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "display_name": "Ms. tester.client.0", 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "email": "tester.client.0_test@test.test", 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "suspended": 0, 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "max_buckets": 1000, 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "subusers": [], 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "keys": [ 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "user": "tester.client.0", 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "access_key": "PGHYMMXPJITOUTKGJVBK", 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "secret_key": "MFCwdw4hIab/pucALqgAArBLuj8b4aBks1GmfdRcNLmojZINX8TJuw==", 2024-03-04T18:07:23.391 INFO:teuthology.orchestra.run.smithi047.stdout: "active": true 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "swift_keys": [], 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": [], 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "default_placement": "", 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "default_storage_class": "", 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "placement_tags": [], 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "bucket_quota": { 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:07:23.392 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: "user_quota": { 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: "temp_url_keys": [], 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "rgw", 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: "mfa_ids": [] 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-04T18:07:23.393 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:23.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.392+0000 7f2048fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:07:23.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.396+0000 7f219460f980 20 remove_watcher() i=0 2024-03-04T18:07:23.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.396+0000 7f219460f980 2 removed watcher, disabling cache 2024-03-04T18:07:23.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.400+0000 7f219460f980 20 remove_watcher() i=1 2024-03-04T18:07:23.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.404+0000 7f219460f980 20 remove_watcher() i=2 2024-03-04T18:07:23.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.404+0000 7f219460f980 20 remove_watcher() i=3 2024-03-04T18:07:23.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.408+0000 7f219460f980 20 remove_watcher() i=4 2024-03-04T18:07:23.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.408+0000 7f219460f980 20 remove_watcher() i=5 2024-03-04T18:07:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.412+0000 7f219460f980 20 remove_watcher() i=6 2024-03-04T18:07:23.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.416+0000 7f219460f980 20 remove_watcher() i=7 2024-03-04T18:07:23.433 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T18:07:23.433 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2024-03-04T18:07:23.440 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2024-03-04T18:07:23.441 DEBUG:teuthology.orchestra.run.smithi047:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2024-03-04T18:07:23.486 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:07:23.560 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:07:23.560 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:07:23.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.608+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.608+0000 7f484eb40980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.608+0000 7f484eb40980 20 realm 2024-03-04T18:07:23.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.608+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.612+0000 7f484eb40980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.612+0000 7f484eb40980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:07:23.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.612+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.612+0000 7f484eb40980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:23.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.612+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.612+0000 7f484eb40980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:07:23.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.612+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:23.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.612+0000 7f484eb40980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:07:23.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.612+0000 7f484eb40980 20 searching for the correct realm 2024-03-04T18:07:23.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.616+0000 7f484c658640 0 -- 172.21.15.47:0/526196403 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f483c0c3040 msgr2=0x7f483c0c81b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:07:23.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:23.828+0000 7f484c658640 0 -- 172.21.15.47:0/526196403 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f4844018040 msgr2=0x7f4844011930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:07:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:07:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:07:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:07:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:07:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:07:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:07:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:07:24.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:07:24.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 zone default found 2024-03-04T18:07:24.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 4 Realm: () 2024-03-04T18:07:24.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:07:24.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:07:24.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:07:24.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 zonegroup default 2024-03-04T18:07:24.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.048+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.052+0000 7f484eb40980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.052+0000 7f484eb40980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.052+0000 7f484eb40980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.052+0000 7f484eb40980 20 started sync module instance, tier type = 2024-03-04T18:07:24.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.052+0000 7f484eb40980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:07:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.076+0000 7f484eb40980 20 add_watcher() i=0 2024-03-04T18:07:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.076+0000 7f484eb40980 20 add_watcher() i=1 2024-03-04T18:07:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.076+0000 7f484eb40980 20 add_watcher() i=2 2024-03-04T18:07:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.076+0000 7f484eb40980 20 add_watcher() i=3 2024-03-04T18:07:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.076+0000 7f484eb40980 20 add_watcher() i=4 2024-03-04T18:07:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.076+0000 7f484eb40980 20 add_watcher() i=5 2024-03-04T18:07:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.076+0000 7f484eb40980 20 add_watcher() i=6 2024-03-04T18:07:24.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.076+0000 7f484eb40980 20 add_watcher() i=7 2024-03-04T18:07:24.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.076+0000 7f484eb40980 2 all 8 watchers are set, enabling cache 2024-03-04T18:07:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.084+0000 7f47027e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:07:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.084+0000 7f484eb40980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:07:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.084+0000 7f484eb40980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:07:24.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.180+0000 7f47017e2640 20 reqs_thread_entry: start 2024-03-04T18:07:24.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.180+0000 7f484eb40980 20 init_complete bucket index max shards: 11 2024-03-04T18:07:24.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.180+0000 7f484eb40980 20 Filter name: none 2024-03-04T18:07:24.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.180+0000 7f4672ffd640 20 reqs_thread_entry: start 2024-03-04T18:07:24.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.180+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:07:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.184+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.188+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.188+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.192+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:07:24.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.196+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.196+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.196+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.196+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_avR_g3LXlAmO90aYmxtiTVZQ-IeS6Kj', time=2024-03-04T20:02:41.736241+0000, chain.objs.size()=2 2024-03-04T18:07:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.196+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.smgSYQLtElJOeW3MpE-fSrRAQobPkDN_1 2024-03-04T18:07:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.196+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.smgSYQLtElJOeW3MpE-fSrRAQobPkDN_2 2024-03-04T18:07:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.200+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.208+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:07:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.212+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.212+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.212+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.216+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:07:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.220+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.220+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.220+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.224+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:07:24.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:07:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_tCs2AFYIcKe5YYCSG2puNk8SrzQDHSy', time=2024-03-04T20:02:41.553303+0000, chain.objs.size()=2 2024-03-04T18:07:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.iIvWexlfPpdG5UdXzEa5hFa0enCDsVi_0 2024-03-04T18:07:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.iIvWexlfPpdG5UdXzEa5hFa0enCDsVi_1 2024-03-04T18:07:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_w2BG5vXwycBV2zM3g_FT_z5vCB_k-j5', time=2024-03-04T20:02:41.715090+0000, chain.objs.size()=3 2024-03-04T18:07:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zb3gqOwuD2dYLT7cpKJceLnc4DtkVGG_0 2024-03-04T18:07:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zb3gqOwuD2dYLT7cpKJceLnc4DtkVGG_1 2024-03-04T18:07:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.228+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zb3gqOwuD2dYLT7cpKJceLnc4DtkVGG_2 2024-03-04T18:07:24.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.240+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.244+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:07:24.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:07:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.1739080031530623205', time=2024-03-04T20:02:41.733322+0000, chain.objs.size()=3 2024-03-04T18:07:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.1 2024-03-04T18:07:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.2 2024-03-04T18:07:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.3 2024-03-04T18:07:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1_1_0a4854d21ffe13c6750b99a9f117ed7a', time=2024-03-04T20:02:41.771611+0000, chain.objs.size()=5 2024-03-04T18:07:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1 2024-03-04T18:07:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1_1 2024-03-04T18:07:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.2 2024-03-04T18:07:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.2_1 2024-03-04T18:07:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.248+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.3 2024-03-04T18:07:24.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.252+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.260+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:07:24.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.264+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.264+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.264+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.264+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_Q3xcVOoqdG5qsL8FnCzAXR5iIbzLqwN', time=2024-03-04T20:02:41.704480+0000, chain.objs.size()=3 2024-03-04T18:07:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.264+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zegKftU_JlKUfv_KCOPSMH0qaVAJIFu_0 2024-03-04T18:07:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.264+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zegKftU_JlKUfv_KCOPSMH0qaVAJIFu_1 2024-03-04T18:07:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.264+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.zegKftU_JlKUfv_KCOPSMH0qaVAJIFu_2 2024-03-04T18:07:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.272+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.276+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:07:24.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.280+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.284+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.284+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.284+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_gPqqm_l0B9XgGBwuy_a2xGcjviGZ_Uz', time=2024-03-04T20:02:41.532400+0000, chain.objs.size()=1 2024-03-04T18:07:24.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.284+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.4aEqhPJAnvyypGuxfbWx98cUiJpQpOf_0 2024-03-04T18:07:24.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.288+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.292+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:07:24.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.296+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.296+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.300+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.300+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_wTykr_axHiR9fiAMD-UX47L12hTOKpT', time=2024-03-04T20:02:41.683866+0000, chain.objs.size()=1 2024-03-04T18:07:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.300+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.YZVxQvz917Gfc6is0xT5L3aLBMaH7HO_0 2024-03-04T18:07:24.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.300+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.308+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:07:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:07:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.14005680552609425226', time=2024-03-04T20:02:41.551007+0000, chain.objs.size()=2 2024-03-04T18:07:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_0 2024-03-04T18:07:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_1 2024-03-04T18:07:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.5446563486005080308', time=2024-03-04T20:02:41.912703+0000, chain.objs.size()=3 2024-03-04T18:07:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.bP1m0jH0-aUQnN_ObtJHiBgKjM4PSM3_0 2024-03-04T18:07:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.bP1m0jH0-aUQnN_ObtJHiBgKjM4PSM3_1 2024-03-04T18:07:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.312+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.bP1m0jH0-aUQnN_ObtJHiBgKjM4PSM3_2 2024-03-04T18:07:24.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.320+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.328+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:07:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:07:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_Q2s0T_CZ-BbrSJuTZ5Fip8uP_Ppd4Xd', time=2024-03-04T20:02:41.578587+0000, chain.objs.size()=1 2024-03-04T18:07:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.WoKRM9-ZC5eihC3R41EczHbaS6OUOQm_1 2024-03-04T18:07:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_qeyxFPzw0wZ9pbj4NJjctv4nZFghtFm', time=2024-03-04T20:02:41.684815+0000, chain.objs.size()=2 2024-03-04T18:07:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.4dH8ROPc6W-kfP0Yew_niVHlRkLpd5K_1 2024-03-04T18:07:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.4dH8ROPc6W-kfP0Yew_niVHlRkLpd5K_2 2024-03-04T18:07:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.6897579346510462014', time=2024-03-04T20:02:41.833817+0000, chain.objs.size()=3 2024-03-04T18:07:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-STANDARD.2~oNqtygdxtILu9rgeuNSwdKSplrTRmFw.1 2024-03-04T18:07:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-STANDARD.2~oNqtygdxtILu9rgeuNSwdKSplrTRmFw.2 2024-03-04T18:07:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.332+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-STANDARD.2~oNqtygdxtILu9rgeuNSwdKSplrTRmFw.3 2024-03-04T18:07:24.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.348+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.352+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:07:24.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.356+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.360+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.360+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.360+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.2_ee25a3e7762809002588a4ae7bc98789', time=2024-03-04T20:02:41.762486+0000, chain.objs.size()=3 2024-03-04T18:07:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.360+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.1 2024-03-04T18:07:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.360+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.2 2024-03-04T18:07:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.360+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-STANDARD.2~ThD2wUN1YoE7SED4ULuzXengJeCelZt.3 2024-03-04T18:07:24.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.364+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.368+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:07:24.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.380+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.380+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.380+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.380+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.13102855747637122916', time=2024-03-04T20:02:41.366734+0000, chain.objs.size()=4 2024-03-04T18:07:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.380+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.12__multipart_foo.2~lp4wl5liFv-J_rfgX_9OgtQRAx3ofnK.1 2024-03-04T18:07:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.380+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.12__multipart_foo.2~lp4wl5liFv-J_rfgX_9OgtQRAx3ofnK.2 2024-03-04T18:07:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.380+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.12__multipart_foo.2~lp4wl5liFv-J_rfgX_9OgtQRAx3ofnK.3 2024-03-04T18:07:24.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.380+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.12__multipart_foo.2~lp4wl5liFv-J_rfgX_9OgtQRAx3ofnK.4 2024-03-04T18:07:24.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.388+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.392+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:07:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.396+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:07:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.9789271116981981130', time=2024-03-04T20:02:41.420701+0000, chain.objs.size()=4 2024-03-04T18:07:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.10__multipart_foo.2~o-nCrA4GpdQ2cUm9wefoFtvV4H8xKhL.1 2024-03-04T18:07:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.10__multipart_foo.2~o-nCrA4GpdQ2cUm9wefoFtvV4H8xKhL.2 2024-03-04T18:07:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.10__multipart_foo.2~o-nCrA4GpdQ2cUm9wefoFtvV4H8xKhL.3 2024-03-04T18:07:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.10__multipart_foo.2~o-nCrA4GpdQ2cUm9wefoFtvV4H8xKhL.4 2024-03-04T18:07:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_6iG3oCpQd_gBS6npn802tsXSuTX9Rdq', time=2024-03-04T20:02:41.615036+0000, chain.objs.size()=1 2024-03-04T18:07:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.dRmV9MS-L3--wEXNaN0P5Tn4ekbEUxC_0 2024-03-04T18:07:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_fug73i_hhm41yXv34zkblxchvQXjvob', time=2024-03-04T20:02:41.774117+0000, chain.objs.size()=1 2024-03-04T18:07:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.400+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_.ElTGv4y1Va1jJ_ngWtnkEdhNs_s3Xw5_0 2024-03-04T18:07:24.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.408+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.412+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:07:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.416+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.416+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.420+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.420+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_1_06360b9bc047dd7174c6c53cb544f79b', time=2024-03-04T20:02:41.612261+0000, chain.objs.size()=2 2024-03-04T18:07:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.420+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_0 2024-03-04T18:07:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.420+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_1 2024-03-04T18:07:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.420+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.428+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:07:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.432+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.432+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.432+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.436+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:07:24.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:07:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.9679328132611048601', time=2024-03-04T20:02:41.476833+0000, chain.objs.size()=4 2024-03-04T18:07:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.9__multipart_foo.2~2KqC2PNgWr40jRqZVhF8UQumGSCoeDH.1 2024-03-04T18:07:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.9__multipart_foo.2~2KqC2PNgWr40jRqZVhF8UQumGSCoeDH.2 2024-03-04T18:07:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.9__multipart_foo.2~2KqC2PNgWr40jRqZVhF8UQumGSCoeDH.3 2024-03-04T18:07:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.9__multipart_foo.2~2KqC2PNgWr40jRqZVhF8UQumGSCoeDH.4 2024-03-04T18:07:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_LKTUjfqrj98u7_d8vMM9YQuJTOv_sRt', time=2024-03-04T20:02:41.562616+0000, chain.objs.size()=2 2024-03-04T18:07:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.CfwNpRssXVb2FlZB9zdy37552CpbuuJ_0 2024-03-04T18:07:24.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.440+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.CfwNpRssXVb2FlZB9zdy37552CpbuuJ_1 2024-03-04T18:07:24.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.448+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.456+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:07:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.456+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.460+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.460+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.464+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:07:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.464+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.468+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.468+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.468+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.6381786571596053396', time=2024-03-04T20:02:41.589143+0000, chain.objs.size()=2 2024-03-04T18:07:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.468+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_0 2024-03-04T18:07:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.468+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.n5LNCDyKAwyU3y2gc2XWyJexn5dM9Ot_1 2024-03-04T18:07:24.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.472+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.476+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:07:24.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.480+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.480+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.480+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.484+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:07:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.488+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.488+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.488+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.488+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='_YOv8w7R6AywxdAI1h_wT9nRvpK0DSgu', time=2024-03-04T20:02:41.541643+0000, chain.objs.size()=1 2024-03-04T18:07:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.488+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.-YGKFu-omL6wq4yRHliGfuEKMIKSgNZ_1 2024-03-04T18:07:24.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.492+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.500+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:07:24.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.504+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.504+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.508+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.508+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:07:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.512+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:07:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_1_eddb828882091a3d9c8cf3afdac2b7cf', time=2024-03-04T20:02:41.632800+0000, chain.objs.size()=2 2024-03-04T18:07:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_0 2024-03-04T18:07:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.15__shadow_.hBpj5D3zg_r0jTwuOykvZJvOQ7qb-hy_1 2024-03-04T18:07:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.13233331196442993742', time=2024-03-04T20:02:41.832645+0000, chain.objs.size()=5 2024-03-04T18:07:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.1 2024-03-04T18:07:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.14__shadow_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.1_1 2024-03-04T18:07:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.2 2024-03-04T18:07:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.14__shadow_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.2_1 2024-03-04T18:07:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.516+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-FROZEN.2~l_LKMmiU0hWt4_jhLmlatwk-ZmMVflM.3 2024-03-04T18:07:24.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.524+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.528+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:07:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.532+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.532+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.532+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.532+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.13624126864927501956', time=2024-03-04T20:02:41.890418+0000, chain.objs.size()=3 2024-03-04T18:07:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.532+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.1wJF164StEx4we8RN9joPNXbJudUpyq_0 2024-03-04T18:07:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.532+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.1wJF164StEx4we8RN9joPNXbJudUpyq_1 2024-03-04T18:07:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.532+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.13__shadow_.1wJF164StEx4we8RN9joPNXbJudUpyq_2 2024-03-04T18:07:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.540+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.544+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:07:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.548+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.552+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.552+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.556+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:07:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.560+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.560+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.560+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.560+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.16872373066293482154', time=2024-03-04T20:02:41.843050+0000, chain.objs.size()=5 2024-03-04T18:07:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.560+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.1 2024-03-04T18:07:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.560+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.14__shadow_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.1_1 2024-03-04T18:07:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.560+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.2 2024-03-04T18:07:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.560+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.14__shadow_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.2_1 2024-03-04T18:07:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.560+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.14__multipart_foo-LUKEWARM.2~3uaDMRrie3csvlStOoBY-oFL_HVTHb6.3 2024-03-04T18:07:24.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.568+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.576+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:07:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.580+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.580+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.580+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.584+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:07:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.588+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.588+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.588+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.588+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1_1_bb7763219b770929e6480b31170d9ca1', time=2024-03-04T20:02:41.783714+0000, chain.objs.size()=5 2024-03-04T18:07:24.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.588+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1 2024-03-04T18:07:24.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.588+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1_1 2024-03-04T18:07:24.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.588+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.2 2024-03-04T18:07:24.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.588+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.2_1 2024-03-04T18:07:24.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.588+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.3 2024-03-04T18:07:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.592+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.600+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:07:24.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.604+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.604+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.608+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.608+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:07:24.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.612+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.616+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.616+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.616+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.2877775117125025815', time=2024-03-04T20:02:41.324904+0000, chain.objs.size()=4 2024-03-04T18:07:24.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.616+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.11__multipart_foo.2~gaCleU2Mrf7rGG3WQ95WLbIpLpDWAVP.1 2024-03-04T18:07:24.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.616+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.11__multipart_foo.2~gaCleU2Mrf7rGG3WQ95WLbIpLpDWAVP.2 2024-03-04T18:07:24.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.616+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.11__multipart_foo.2~gaCleU2Mrf7rGG3WQ95WLbIpLpDWAVP.3 2024-03-04T18:07:24.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.616+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.11__multipart_foo.2~gaCleU2Mrf7rGG3WQ95WLbIpLpDWAVP.4 2024-03-04T18:07:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.624+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.628+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:07:24.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.632+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.632+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.636+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.636+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.18141961772719482821', time=2024-03-04T20:02:41.694177+0000, chain.objs.size()=5 2024-03-04T18:07:24.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.636+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1 2024-03-04T18:07:24.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.636+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.1_1 2024-03-04T18:07:24.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.636+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.2 2024-03-04T18:07:24.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.636+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.2_1 2024-03-04T18:07:24.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.636+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-FROZEN.2~6nCa7H7vmeaKVha7d7FK0ymfLs6ZM3B.3 2024-03-04T18:07:24.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.640+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.644+0000 7f484eb40980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:07:24.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.648+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.648+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.652+0000 7f484eb40980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:07:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.652+0000 7f484eb40980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.6702748022887742919', time=2024-03-04T20:02:41.752608+0000, chain.objs.size()=5 2024-03-04T18:07:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.652+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1 2024-03-04T18:07:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.652+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.1_1 2024-03-04T18:07:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.652+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.2 2024-03-04T18:07:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.652+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__shadow_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.2_1 2024-03-04T18:07:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.652+0000 7f484eb40980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:797b67a4-b876-4d04-b122-10afd858d465.4453.16__multipart_foo-LUKEWARM.2~4qMNxDdC5Sr7dg-R-QqZIBinPboGqh6.3 2024-03-04T18:07:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.652+0000 7f484eb40980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:07:24.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.660+0000 7f47027e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:07:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.668+0000 7f484eb40980 20 remove_watcher() i=0 2024-03-04T18:07:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.668+0000 7f484eb40980 2 removed watcher, disabling cache 2024-03-04T18:07:24.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.668+0000 7f484eb40980 20 remove_watcher() i=1 2024-03-04T18:07:24.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.672+0000 7f484eb40980 20 remove_watcher() i=2 2024-03-04T18:07:24.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.676+0000 7f484eb40980 20 remove_watcher() i=3 2024-03-04T18:07:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.676+0000 7f484eb40980 20 remove_watcher() i=4 2024-03-04T18:07:24.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.680+0000 7f484eb40980 20 remove_watcher() i=5 2024-03-04T18:07:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.684+0000 7f484eb40980 20 remove_watcher() i=6 2024-03-04T18:07:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.688+0000 7f484eb40980 20 remove_watcher() i=7 2024-03-04T18:07:24.705 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:07:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:07:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:07:24.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.828+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.828+0000 7ff899aff980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.828+0000 7ff899aff980 20 realm 2024-03-04T18:07:24.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.828+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.828+0000 7ff899aff980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.828+0000 7ff899aff980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:07:24.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.828+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.832+0000 7ff899aff980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.832+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.832+0000 7ff899aff980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:07:24.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.832+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.832+0000 7ff899aff980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:07:24.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.832+0000 7ff899aff980 20 searching for the correct realm 2024-03-04T18:07:24.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.840+0000 7ff88ee16640 0 -- 172.21.15.47:0/2725503318 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7ff88001c3c0 msgr2=0x7ff88001c840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:07:24.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.844+0000 7ff897617640 0 -- 172.21.15.47:0/2725503318 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7ff8900bf040 msgr2=0x7ff8900d7410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:07:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:07:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:07:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:07:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:07:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:07:24.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:07:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:07:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.852+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:07:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 zone default found 2024-03-04T18:07:24.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 4 Realm: () 2024-03-04T18:07:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:07:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:07:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:07:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 zonegroup default 2024-03-04T18:07:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 rados->read ofs=0 len=0 2024-03-04T18:07:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:07:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 started sync module instance, tier type = 2024-03-04T18:07:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.856+0000 7ff899aff980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:07:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.872+0000 7ff899aff980 20 add_watcher() i=0 2024-03-04T18:07:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.872+0000 7ff899aff980 20 add_watcher() i=1 2024-03-04T18:07:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.872+0000 7ff899aff980 20 add_watcher() i=2 2024-03-04T18:07:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.872+0000 7ff899aff980 20 add_watcher() i=3 2024-03-04T18:07:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.872+0000 7ff899aff980 20 add_watcher() i=4 2024-03-04T18:07:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.872+0000 7ff899aff980 20 add_watcher() i=5 2024-03-04T18:07:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.872+0000 7ff899aff980 20 add_watcher() i=6 2024-03-04T18:07:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.876+0000 7ff899aff980 20 add_watcher() i=7 2024-03-04T18:07:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.876+0000 7ff899aff980 2 all 8 watchers are set, enabling cache 2024-03-04T18:07:24.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.880+0000 7ff74dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:07:24.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.880+0000 7ff899aff980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:07:24.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.880+0000 7ff899aff980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:07:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.960+0000 7ff74cfe1640 20 reqs_thread_entry: start 2024-03-04T18:07:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.960+0000 7ff899aff980 20 init_complete bucket index max shards: 11 2024-03-04T18:07:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.960+0000 7ff899aff980 20 Filter name: none 2024-03-04T18:07:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.960+0000 7ff6c67fc640 20 reqs_thread_entry: start 2024-03-04T18:07:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.960+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:07:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.964+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.968+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.968+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.980+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:07:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.980+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.984+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.984+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.988+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:07:24.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.992+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.992+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.992+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:24.996+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:07:24.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.000+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:24.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.000+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:24.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.000+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.000+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:07:25.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.004+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.008+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.008+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.008+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:07:25.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.012+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.012+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.012+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.016+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:07:25.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.020+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.020+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.020+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.024+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:07:25.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.028+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.028+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.028+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.028+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:07:25.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.032+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.032+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.036+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.036+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:07:25.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.040+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.040+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.040+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.044+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:07:25.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.048+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.048+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.052+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.052+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:07:25.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.056+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.056+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.056+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.060+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:07:25.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.064+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.064+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.064+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.068+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:07:25.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.072+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.072+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.076+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.076+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:07:25.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.080+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.080+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.080+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.084+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:07:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.088+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.088+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.088+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.088+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:07:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.092+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.096+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.096+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.100+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:07:25.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.100+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.104+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.104+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.108+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:07:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.108+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.112+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.112+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.116+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:07:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.116+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.120+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.120+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.120+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:07:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.124+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.128+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.128+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.128+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:07:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.132+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.136+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.136+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.140+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:07:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.144+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.148+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.148+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.148+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:07:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.152+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.156+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.156+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.160+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:07:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.164+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.164+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.168+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.168+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:07:25.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.172+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.176+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.176+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.180+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:07:25.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.184+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.184+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.184+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.188+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:07:25.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.192+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.192+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.196+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.196+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:07:25.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.200+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.204+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.204+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.208+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:07:25.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.212+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.212+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.212+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.216+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:07:25.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.220+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.220+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.224+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.224+0000 7ff899aff980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:07:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.228+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.228+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:07:25.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.232+0000 7ff899aff980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:07:25.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.236+0000 7ff74dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:07:25.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.240+0000 7ff899aff980 20 remove_watcher() i=0 2024-03-04T18:07:25.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.240+0000 7ff899aff980 2 removed watcher, disabling cache 2024-03-04T18:07:25.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.244+0000 7ff899aff980 20 remove_watcher() i=1 2024-03-04T18:07:25.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.248+0000 7ff899aff980 20 remove_watcher() i=2 2024-03-04T18:07:25.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.248+0000 7ff899aff980 20 remove_watcher() i=3 2024-03-04T18:07:25.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.252+0000 7ff899aff980 20 remove_watcher() i=4 2024-03-04T18:07:25.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.256+0000 7ff899aff980 20 remove_watcher() i=5 2024-03-04T18:07:25.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.260+0000 7ff899aff980 20 remove_watcher() i=6 2024-03-04T18:07:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:07:25.260+0000 7ff899aff980 20 remove_watcher() i=7 2024-03-04T18:07:25.287 DEBUG:teuthology.orchestra.run.smithi047:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2024-03-04T18:07:25.913 INFO:teuthology.orchestra.run.smithi047.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2024-03-04T18:07:31.015 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :clean UP-TO-DATE 2024-03-04T18:07:54.513 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :compileJava 2024-03-04T18:07:54.613 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :processResources 2024-03-04T18:07:54.613 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :classes 2024-03-04T18:07:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:55.213 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :compileTestJava 2024-03-04T18:07:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-03-04T18:07:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-03-04T18:07:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:07:55.215 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :processTestResources NO-SOURCE 2024-03-04T18:07:55.215 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :testClasses 2024-03-04T18:08:02.013 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:02.013 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :test 2024-03-04T18:08:02.014 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:02.014 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2024-03-04T18:08:14.513 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:14.513 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2024-03-04T18:08:15.313 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:15.313 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2024-03-04T18:08:15.513 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:15.513 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2024-03-04T18:08:16.313 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:16.313 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2024-03-04T18:08:38.613 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:38.614 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2024-03-04T18:08:39.613 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:39.613 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2024-03-04T18:08:39.713 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:39.713 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2024-03-04T18:08:39.812 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:39.813 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2024-03-04T18:08:40.013 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2024-03-04T18:08:40.813 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:08:40.813 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2024-03-04T18:09:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:09:04.314 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2024-03-04T18:09:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:09:05.813 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2024-03-04T18:09:06.313 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:09:06.313 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STANDARD_OUT 2024-03-04T18:09:06.313 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04 18:09:06,274 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-03-04T18:10:10.714 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:10.714 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2024-03-04T18:10:12.213 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:12.213 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2024-03-04T18:10:12.413 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:12.413 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2024-03-04T18:10:13.213 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:13.213 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2024-03-04T18:10:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:13.513 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2024-03-04T18:10:14.813 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:14.813 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2024-03-04T18:10:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:28.513 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2024-03-04T18:10:29.313 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:29.313 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2024-03-04T18:10:29.413 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:29.413 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2024-03-04T18:10:29.613 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:29.613 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2024-03-04T18:10:58.714 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:10:58.714 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2024-03-04T18:11:02.413 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:11:02.413 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests STANDARD_OUT 2024-03-04T18:11:02.413 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04 18:11:02,390 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:11:02.512 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04 18:11:02,414 [s3-transfer-manager-worker-10] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:11:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:11:04.613 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2024-03-04T18:11:37.213 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:11:37.213 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2024-03-04T18:11:38.412 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:11:38.413 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2024-03-04T18:11:49.613 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:11:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2024-03-04T18:11:50.412 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:11:50.413 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2024-03-04T18:15:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:05.117 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2024-03-04T18:15:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:05.915 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2024-03-04T18:15:23.316 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:23.316 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2024-03-04T18:15:24.115 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:24.116 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2024-03-04T18:15:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:37.116 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2024-03-04T18:15:37.916 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:37.916 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2024-03-04T18:15:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:42.816 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2024-03-04T18:15:55.716 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:55.716 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2024-03-04T18:15:56.516 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:56.516 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2024-03-04T18:15:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:59.716 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2024-03-04T18:15:59.916 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:15:59.916 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2024-03-04T18:16:00.716 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:00.716 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2024-03-04T18:16:00.916 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:00.916 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2024-03-04T18:16:01.816 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:01.816 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2024-03-04T18:16:02.016 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:02.016 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2024-03-04T18:16:02.916 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:02.916 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2024-03-04T18:16:03.116 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:03.116 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2024-03-04T18:16:04.016 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:04.016 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2024-03-04T18:16:04.216 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:04.216 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2024-03-04T18:16:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:05.116 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2024-03-04T18:16:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:05.316 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2024-03-04T18:16:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:06.116 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2024-03-04T18:16:36.117 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:36.117 INFO:teuthology.orchestra.run.smithi047.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2024-03-04T18:16:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:37.117 INFO:teuthology.orchestra.run.smithi047.stdout:---------------------------------------------------------------------- 2024-03-04T18:16:37.118 INFO:teuthology.orchestra.run.smithi047.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2024-03-04T18:16:37.118 INFO:teuthology.orchestra.run.smithi047.stdout:---------------------------------------------------------------------- 2024-03-04T18:16:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:37.118 INFO:teuthology.orchestra.run.smithi047.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-03-04T18:16:37.118 INFO:teuthology.orchestra.run.smithi047.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-03-04T18:16:37.118 INFO:teuthology.orchestra.run.smithi047.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-03-04T18:16:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:37.118 INFO:teuthology.orchestra.run.smithi047.stdout:BUILD SUCCESSFUL in 9m 11s 2024-03-04T18:16:37.118 INFO:teuthology.orchestra.run.smithi047.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2024-03-04T18:16:37.449 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:16:37.481 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:16:37.482 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:16:37.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.524+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:37.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 realm 2024-03-04T18:16:37.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:37.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:16:37.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:37.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:37.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:16:37.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.528+0000 7fc099a02980 20 searching for the correct realm 2024-03-04T18:16:37.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.536+0000 7fc097d1b640 0 -- 172.21.15.47:0/2863677447 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55fcf9c36e60 msgr2=0x55fcf9c39320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.544+0000 7fc09751a640 0 -- 172.21.15.47:0/2863677447 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x55fcf9c3b240 msgr2=0x55fcf9c3d630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:16:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:16:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:16:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:16:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:16:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:16:37.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:37.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:37.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:16:37.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 zone default found 2024-03-04T18:16:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 4 Realm: () 2024-03-04T18:16:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:16:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:16:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:16:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 zonegroup default 2024-03-04T18:16:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:37.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.552+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.556+0000 7fc099a02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.556+0000 7fc099a02980 20 rados->read ofs=0 len=0 2024-03-04T18:16:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.556+0000 7fc099a02980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.556+0000 7fc099a02980 20 started sync module instance, tier type = 2024-03-04T18:16:37.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.556+0000 7fc099a02980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:16:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.572+0000 7fc099a02980 20 add_watcher() i=0 2024-03-04T18:16:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.572+0000 7fc099a02980 20 add_watcher() i=1 2024-03-04T18:16:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.572+0000 7fc099a02980 20 add_watcher() i=2 2024-03-04T18:16:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.572+0000 7fc099a02980 20 add_watcher() i=3 2024-03-04T18:16:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.572+0000 7fc099a02980 20 add_watcher() i=4 2024-03-04T18:16:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.572+0000 7fc099a02980 20 add_watcher() i=5 2024-03-04T18:16:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.572+0000 7fc099a02980 20 add_watcher() i=6 2024-03-04T18:16:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.576+0000 7fc099a02980 20 add_watcher() i=7 2024-03-04T18:16:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.576+0000 7fc099a02980 2 all 8 watchers are set, enabling cache 2024-03-04T18:16:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.580+0000 7fbf4dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.580+0000 7fc099a02980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:16:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.580+0000 7fc099a02980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:16:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.664+0000 7fbf4cfe1640 20 reqs_thread_entry: start 2024-03-04T18:16:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.664+0000 7fc099a02980 20 init_complete bucket index max shards: 11 2024-03-04T18:16:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.664+0000 7fc099a02980 20 Filter name: none 2024-03-04T18:16:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.664+0000 7fbec67fc640 20 reqs_thread_entry: start 2024-03-04T18:16:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.664+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:16:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.668+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.668+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.668+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.668+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:16:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.672+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.676+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.676+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.676+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:16:37.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.680+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.680+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.680+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.684+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:16:37.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:16:37.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.13772250362934679263', time=2024-03-04T20:09:51.237518+0000, chain.objs.size()=9 2024-03-04T18:16:37.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~LlpImlUlRFqlX9df7jTjNWndiA3s5sh.1 2024-03-04T18:16:37.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~LlpImlUlRFqlX9df7jTjNWndiA3s5sh.1_1 2024-03-04T18:16:37.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~LlpImlUlRFqlX9df7jTjNWndiA3s5sh.2 2024-03-04T18:16:37.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~LlpImlUlRFqlX9df7jTjNWndiA3s5sh.2_1 2024-03-04T18:16:37.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~LlpImlUlRFqlX9df7jTjNWndiA3s5sh.3 2024-03-04T18:16:37.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~LlpImlUlRFqlX9df7jTjNWndiA3s5sh.3_1 2024-03-04T18:16:37.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~LlpImlUlRFqlX9df7jTjNWndiA3s5sh.4 2024-03-04T18:16:37.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~LlpImlUlRFqlX9df7jTjNWndiA3s5sh.4_1 2024-03-04T18:16:37.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~LlpImlUlRFqlX9df7jTjNWndiA3s5sh.5 2024-03-04T18:16:37.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.4870035479087774033', time=2024-03-04T20:10:28.690111+0000, chain.objs.size()=9 2024-03-04T18:16:37.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.40__multipart_key1.2~Ji8O_KpDajP3iYU4AKdyaJiwfxDWNqE.1 2024-03-04T18:16:37.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.40__shadow_key1.2~Ji8O_KpDajP3iYU4AKdyaJiwfxDWNqE.1_1 2024-03-04T18:16:37.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.688+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.40__multipart_key1.2~Ji8O_KpDajP3iYU4AKdyaJiwfxDWNqE.2 2024-03-04T18:16:37.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.696+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.40__shadow_key1.2~Ji8O_KpDajP3iYU4AKdyaJiwfxDWNqE.2_1 2024-03-04T18:16:37.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.696+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.40__multipart_key1.2~Ji8O_KpDajP3iYU4AKdyaJiwfxDWNqE.3 2024-03-04T18:16:37.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.700+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.40__shadow_key1.2~Ji8O_KpDajP3iYU4AKdyaJiwfxDWNqE.3_1 2024-03-04T18:16:37.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.700+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.40__multipart_key1.2~Ji8O_KpDajP3iYU4AKdyaJiwfxDWNqE.4 2024-03-04T18:16:37.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.700+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.40__shadow_key1.2~Ji8O_KpDajP3iYU4AKdyaJiwfxDWNqE.4_1 2024-03-04T18:16:37.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.700+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.40__multipart_key1.2~Ji8O_KpDajP3iYU4AKdyaJiwfxDWNqE.5 2024-03-04T18:16:37.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.712+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.720+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:16:37.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.724+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.724+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.724+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.728+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:16:37.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.728+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.728+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:16:37.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.12299936922237913096', time=2024-03-04T20:09:40.162294+0000, chain.objs.size()=9 2024-03-04T18:16:37.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~ymmksrOdkmh2HPfSbWJsHcaMiWMKGFz.1 2024-03-04T18:16:37.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~ymmksrOdkmh2HPfSbWJsHcaMiWMKGFz.1_1 2024-03-04T18:16:37.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~ymmksrOdkmh2HPfSbWJsHcaMiWMKGFz.2 2024-03-04T18:16:37.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~ymmksrOdkmh2HPfSbWJsHcaMiWMKGFz.2_1 2024-03-04T18:16:37.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~ymmksrOdkmh2HPfSbWJsHcaMiWMKGFz.3 2024-03-04T18:16:37.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~ymmksrOdkmh2HPfSbWJsHcaMiWMKGFz.3_1 2024-03-04T18:16:37.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~ymmksrOdkmh2HPfSbWJsHcaMiWMKGFz.4 2024-03-04T18:16:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~ymmksrOdkmh2HPfSbWJsHcaMiWMKGFz.4_1 2024-03-04T18:16:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~ymmksrOdkmh2HPfSbWJsHcaMiWMKGFz.5 2024-03-04T18:16:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.15634641904167054160', time=2024-03-04T20:11:37.439901+0000, chain.objs.size()=9 2024-03-04T18:16:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file.mpg.2~8RU0EGALTTFwYoS9D2-omBp-ol7s3ZH.1 2024-03-04T18:16:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__shadow_file.mpg.2~8RU0EGALTTFwYoS9D2-omBp-ol7s3ZH.1_1 2024-03-04T18:16:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.732+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file.mpg.2~8RU0EGALTTFwYoS9D2-omBp-ol7s3ZH.2 2024-03-04T18:16:37.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.736+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__shadow_file.mpg.2~8RU0EGALTTFwYoS9D2-omBp-ol7s3ZH.2_1 2024-03-04T18:16:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.740+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file.mpg.2~8RU0EGALTTFwYoS9D2-omBp-ol7s3ZH.3 2024-03-04T18:16:37.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.740+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__shadow_file.mpg.2~8RU0EGALTTFwYoS9D2-omBp-ol7s3ZH.3_1 2024-03-04T18:16:37.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.740+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file.mpg.2~8RU0EGALTTFwYoS9D2-omBp-ol7s3ZH.4 2024-03-04T18:16:37.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.740+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__shadow_file.mpg.2~8RU0EGALTTFwYoS9D2-omBp-ol7s3ZH.4_1 2024-03-04T18:16:37.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.740+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file.mpg.2~8RU0EGALTTFwYoS9D2-omBp-ol7s3ZH.5 2024-03-04T18:16:37.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.752+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.756+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:16:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.756+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.760+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.760+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.760+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.6055784487913242076', time=2024-03-04T20:11:37.615305+0000, chain.objs.size()=5 2024-03-04T18:16:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.760+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file2.mpg.2~IbsviXnGQ9ClO4F8ZNrvTbVOP679OsQ.1 2024-03-04T18:16:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.760+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file2.mpg.2~IbsviXnGQ9ClO4F8ZNrvTbVOP679OsQ.2 2024-03-04T18:16:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.760+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file2.mpg.2~IbsviXnGQ9ClO4F8ZNrvTbVOP679OsQ.3 2024-03-04T18:16:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.760+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file2.mpg.2~IbsviXnGQ9ClO4F8ZNrvTbVOP679OsQ.4 2024-03-04T18:16:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.760+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.42__multipart_file2.mpg.2~IbsviXnGQ9ClO4F8ZNrvTbVOP679OsQ.5 2024-03-04T18:16:37.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.764+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.768+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:16:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.772+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.772+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.776+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.776+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:16:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.2495110890386291785', time=2024-03-04T20:09:56.122487+0000, chain.objs.size()=7 2024-03-04T18:16:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~f2Cck-Eg7U0ibFPMBOKndUSMbSlrArH.1 2024-03-04T18:16:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~f2Cck-Eg7U0ibFPMBOKndUSMbSlrArH.1_1 2024-03-04T18:16:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~f2Cck-Eg7U0ibFPMBOKndUSMbSlrArH.1_2 2024-03-04T18:16:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~f2Cck-Eg7U0ibFPMBOKndUSMbSlrArH.2 2024-03-04T18:16:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~f2Cck-Eg7U0ibFPMBOKndUSMbSlrArH.2_1 2024-03-04T18:16:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~f2Cck-Eg7U0ibFPMBOKndUSMbSlrArH.2_2 2024-03-04T18:16:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.780+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~f2Cck-Eg7U0ibFPMBOKndUSMbSlrArH.3 2024-03-04T18:16:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.788+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.792+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:16:37.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.796+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.796+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.800+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:16:37.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:37.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.13059770426718441297', time=2024-03-04T20:08:39.020459+0000, chain.objs.size()=9 2024-03-04T18:16:37.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.31__multipart_key1.2~K8BbQZlMhJRRXEAP_BTfd7x7BJbJM9J.1 2024-03-04T18:16:37.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.31__shadow_key1.2~K8BbQZlMhJRRXEAP_BTfd7x7BJbJM9J.1_1 2024-03-04T18:16:37.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.31__multipart_key1.2~K8BbQZlMhJRRXEAP_BTfd7x7BJbJM9J.2 2024-03-04T18:16:37.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.31__shadow_key1.2~K8BbQZlMhJRRXEAP_BTfd7x7BJbJM9J.2_1 2024-03-04T18:16:37.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.31__multipart_key1.2~K8BbQZlMhJRRXEAP_BTfd7x7BJbJM9J.3 2024-03-04T18:16:37.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.31__shadow_key1.2~K8BbQZlMhJRRXEAP_BTfd7x7BJbJM9J.3_1 2024-03-04T18:16:37.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.31__multipart_key1.2~K8BbQZlMhJRRXEAP_BTfd7x7BJbJM9J.4 2024-03-04T18:16:37.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.31__shadow_key1.2~K8BbQZlMhJRRXEAP_BTfd7x7BJbJM9J.4_1 2024-03-04T18:16:37.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.804+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.31__multipart_key1.2~K8BbQZlMhJRRXEAP_BTfd7x7BJbJM9J.5 2024-03-04T18:16:37.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.816+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:16:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.17938912153842668410', time=2024-03-04T20:11:02.025013+0000, chain.objs.size()=9 2024-03-04T18:16:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.41__multipart_key1.2~vqMLLyb7C_0saSaBe3Jg5ubYkk1ZoA6.1 2024-03-04T18:16:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.41__shadow_key1.2~vqMLLyb7C_0saSaBe3Jg5ubYkk1ZoA6.1_1 2024-03-04T18:16:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.41__multipart_key1.2~vqMLLyb7C_0saSaBe3Jg5ubYkk1ZoA6.2 2024-03-04T18:16:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.41__shadow_key1.2~vqMLLyb7C_0saSaBe3Jg5ubYkk1ZoA6.2_1 2024-03-04T18:16:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.41__multipart_key1.2~vqMLLyb7C_0saSaBe3Jg5ubYkk1ZoA6.3 2024-03-04T18:16:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.41__shadow_key1.2~vqMLLyb7C_0saSaBe3Jg5ubYkk1ZoA6.3_1 2024-03-04T18:16:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.41__multipart_key1.2~vqMLLyb7C_0saSaBe3Jg5ubYkk1ZoA6.4 2024-03-04T18:16:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.824+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.41__shadow_key1.2~vqMLLyb7C_0saSaBe3Jg5ubYkk1ZoA6.4_1 2024-03-04T18:16:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.828+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.41__multipart_key1.2~vqMLLyb7C_0saSaBe3Jg5ubYkk1ZoA6.5 2024-03-04T18:16:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.836+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.840+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:16:37.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.840+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:16:37.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.9371385683771415254', time=2024-03-04T20:10:10.600794+0000, chain.objs.size()=7 2024-03-04T18:16:37.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~32c2Wp9-lWQzqeUAysJxmF3cdnWP1ZR.1 2024-03-04T18:16:37.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~32c2Wp9-lWQzqeUAysJxmF3cdnWP1ZR.1_1 2024-03-04T18:16:37.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~32c2Wp9-lWQzqeUAysJxmF3cdnWP1ZR.1_2 2024-03-04T18:16:37.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~32c2Wp9-lWQzqeUAysJxmF3cdnWP1ZR.2 2024-03-04T18:16:37.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~32c2Wp9-lWQzqeUAysJxmF3cdnWP1ZR.2_1 2024-03-04T18:16:37.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~32c2Wp9-lWQzqeUAysJxmF3cdnWP1ZR.2_2 2024-03-04T18:16:37.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~32c2Wp9-lWQzqeUAysJxmF3cdnWP1ZR.3 2024-03-04T18:16:37.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.4549608418251310388', time=2024-03-04T20:12:49.449321+0000, chain.objs.size()=21 2024-03-04T18:16:37.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.1 2024-03-04T18:16:37.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.1_1 2024-03-04T18:16:37.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.2 2024-03-04T18:16:37.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.2_1 2024-03-04T18:16:37.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.844+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.3 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.3_1 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.4 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.4_1 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.5 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.5_1 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.6 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.6_1 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.7 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.7_1 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.8 2024-03-04T18:16:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.852+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.8_1 2024-03-04T18:16:37.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.860+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.9 2024-03-04T18:16:37.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.860+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.9_1 2024-03-04T18:16:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.864+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.10 2024-03-04T18:16:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.864+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.10_1 2024-03-04T18:16:37.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.864+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~fAO8woaPm4TTAaj3Hn_2hAZBqZMDpXm.11 2024-03-04T18:16:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.864+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.3649572054569894872', time=2024-03-04T20:16:36.369596+0000, chain.objs.size()=21 2024-03-04T18:16:37.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.864+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.1 2024-03-04T18:16:37.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.868+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.1_1 2024-03-04T18:16:37.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.868+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.2 2024-03-04T18:16:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.868+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.2_1 2024-03-04T18:16:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.868+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.3 2024-03-04T18:16:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.868+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.3_1 2024-03-04T18:16:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.868+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.4 2024-03-04T18:16:37.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.868+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.4_1 2024-03-04T18:16:37.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.876+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.5 2024-03-04T18:16:37.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.876+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.5_1 2024-03-04T18:16:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.876+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.6 2024-03-04T18:16:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.876+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.6_1 2024-03-04T18:16:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.876+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.7 2024-03-04T18:16:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.876+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.7_1 2024-03-04T18:16:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.880+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.8 2024-03-04T18:16:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.880+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.8_1 2024-03-04T18:16:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.880+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.9 2024-03-04T18:16:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.880+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.9_1 2024-03-04T18:16:37.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.880+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.10 2024-03-04T18:16:37.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.884+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__shadow_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.10_1 2024-03-04T18:16:37.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.888+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.55__multipart_key1.2~Pp2sQwtM9KnbYeUVX9kXDYuBfi_g7BP.11 2024-03-04T18:16:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.892+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.892+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:16:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.2581693266043024160', time=2024-03-04T20:15:04.963742+0000, chain.objs.size()=16 2024-03-04T18:16:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.1 2024-03-04T18:16:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.1_1 2024-03-04T18:16:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.1_2 2024-03-04T18:16:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.2 2024-03-04T18:16:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.2_1 2024-03-04T18:16:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.2_2 2024-03-04T18:16:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.3 2024-03-04T18:16:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.3_1 2024-03-04T18:16:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.3_2 2024-03-04T18:16:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.4 2024-03-04T18:16:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.4_1 2024-03-04T18:16:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.896+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.4_2 2024-03-04T18:16:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.904+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.5 2024-03-04T18:16:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.904+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.5_1 2024-03-04T18:16:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.904+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.5_2 2024-03-04T18:16:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.904+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~g8woR-ZOVAwlqUvqNXgmB5CN-_h5wr-.6 2024-03-04T18:16:37.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.908+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.912+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:16:37.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.916+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.916+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.916+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.916+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:16:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.920+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.920+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.920+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.920+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:16:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.15409048061758317487', time=2024-03-04T20:09:46.314465+0000, chain.objs.size()=9 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~V97H3mddawcvX7Y4hn04uJOC47Wumo5.1 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~V97H3mddawcvX7Y4hn04uJOC47Wumo5.1_1 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~V97H3mddawcvX7Y4hn04uJOC47Wumo5.2 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~V97H3mddawcvX7Y4hn04uJOC47Wumo5.2_1 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~V97H3mddawcvX7Y4hn04uJOC47Wumo5.3 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~V97H3mddawcvX7Y4hn04uJOC47Wumo5.3_1 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~V97H3mddawcvX7Y4hn04uJOC47Wumo5.4 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~V97H3mddawcvX7Y4hn04uJOC47Wumo5.4_1 2024-03-04T18:16:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.924+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~V97H3mddawcvX7Y4hn04uJOC47Wumo5.5 2024-03-04T18:16:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.928+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.932+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:16:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.932+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.932+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.936+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.936+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.2982016567250407179', time=2024-03-04T20:10:10.936045+0000, chain.objs.size()=5 2024-03-04T18:16:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.936+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.35__shadow_.eRVDxYKna5LRo9naqHkMwcmtEP-xr6o_1 2024-03-04T18:16:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.936+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.35__shadow_.eRVDxYKna5LRo9naqHkMwcmtEP-xr6o_2 2024-03-04T18:16:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.936+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.35__shadow_.eRVDxYKna5LRo9naqHkMwcmtEP-xr6o_3 2024-03-04T18:16:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.936+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.35__shadow_.eRVDxYKna5LRo9naqHkMwcmtEP-xr6o_4 2024-03-04T18:16:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.936+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.35__shadow_.eRVDxYKna5LRo9naqHkMwcmtEP-xr6o_5 2024-03-04T18:16:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.940+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.944+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:16:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.944+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.944+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.944+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.948+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:16:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.948+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.948+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.948+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:16:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.33__shadow_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.1_1_1e8b4b29d60b2a68bae831c1977b7513', time=2024-03-04T20:09:04.520801+0000, chain.objs.size()=9 2024-03-04T18:16:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.1 2024-03-04T18:16:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__shadow_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.1_1 2024-03-04T18:16:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.2 2024-03-04T18:16:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__shadow_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.2_1 2024-03-04T18:16:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.3 2024-03-04T18:16:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__shadow_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.3_1 2024-03-04T18:16:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.4 2024-03-04T18:16:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__shadow_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.4_1 2024-03-04T18:16:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.952+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.5 2024-03-04T18:16:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.956+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.960+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:16:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.964+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.964+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.964+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.964+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:16:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.968+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.968+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.968+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.968+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:16:37.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.14308712320752899583', time=2024-03-04T20:09:05.216402+0000, chain.objs.size()=9 2024-03-04T18:16:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.1 2024-03-04T18:16:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__shadow_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.1_1 2024-03-04T18:16:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.2 2024-03-04T18:16:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__shadow_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.2_1 2024-03-04T18:16:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.3 2024-03-04T18:16:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__shadow_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.3_1 2024-03-04T18:16:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.4 2024-03-04T18:16:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__shadow_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.4_1 2024-03-04T18:16:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.972+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.33__multipart_key1.2~N_MOgr74H8V3dcFP2IHTUrlcfbsn1hd.5 2024-03-04T18:16:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.976+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:37.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.976+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:16:37.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:37.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.16921239606609523956', time=2024-03-04T20:13:46.401931+0000, chain.objs.size()=19 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.1 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.1_1 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.2 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.2_1 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.3 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.3_1 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.4 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.4_1 2024-03-04T18:16:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.5 2024-03-04T18:16:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.5_1 2024-03-04T18:16:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.6 2024-03-04T18:16:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.980+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.6_1 2024-03-04T18:16:37.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.984+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.7 2024-03-04T18:16:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.984+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.7_1 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.8 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.8_1 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.9 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.9_1 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~ypwmGfO2-8rnnUFOa6dWsUM8tBLT9YL.10 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI', time=2024-03-04T20:15:52.034137+0000, chain.objs.size()=1 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.16 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI', time=2024-03-04T20:15:52.188692+0000, chain.objs.size()=1 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.17 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI', time=2024-03-04T20:15:52.241587+0000, chain.objs.size()=1 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.20 2024-03-04T18:16:37.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI', time=2024-03-04T20:15:52.407746+0000, chain.objs.size()=37 2024-03-04T18:16:37.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.1 2024-03-04T18:16:37.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.2 2024-03-04T18:16:37.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.988+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.3 2024-03-04T18:16:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.4 2024-03-04T18:16:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.5 2024-03-04T18:16:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.6 2024-03-04T18:16:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.7 2024-03-04T18:16:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.8 2024-03-04T18:16:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.9 2024-03-04T18:16:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.10 2024-03-04T18:16:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.11 2024-03-04T18:16:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.12 2024-03-04T18:16:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.13 2024-03-04T18:16:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:37.996+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.14 2024-03-04T18:16:38.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.000+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.15 2024-03-04T18:16:38.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.000+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.Ec9q0oHohfDmBEDOv_uHoifmWi9q1a3c.16 2024-03-04T18:16:38.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.000+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.Np730Jar3o7lDD0NUEk5BLPx572gmw6Y.17 2024-03-04T18:16:38.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.000+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.18 2024-03-04T18:16:38.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.000+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.19 2024-03-04T18:16:38.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.000+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.KfFVtWhyP96YaqW5erFAKFh7Hymtp-Wy.20 2024-03-04T18:16:38.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.000+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.21 2024-03-04T18:16:38.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.000+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.22 2024-03-04T18:16:38.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.000+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.23 2024-03-04T18:16:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.004+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.24 2024-03-04T18:16:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.004+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.25 2024-03-04T18:16:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.008+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.26 2024-03-04T18:16:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.008+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.27 2024-03-04T18:16:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.008+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.28 2024-03-04T18:16:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.008+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.29 2024-03-04T18:16:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.008+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.30 2024-03-04T18:16:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.008+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.31 2024-03-04T18:16:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.008+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.32 2024-03-04T18:16:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.008+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.33 2024-03-04T18:16:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.008+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.34 2024-03-04T18:16:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.012+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.35 2024-03-04T18:16:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.012+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.36 2024-03-04T18:16:38.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.016+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.47__multipart_key1.2~XkSjpH2dg0r5P0wboyviKMgWCfVfqnI.37 2024-03-04T18:16:38.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.020+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:38.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.020+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:16:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.024+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.024+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.024+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:16:38.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB', time=2024-03-04T20:08:14.426565+0000, chain.objs.size()=9 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.29__multipart_key1.2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB.1 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.29__shadow_key1.2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB.1_1 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.29__multipart_key1.2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB.2 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.29__shadow_key1.2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB.2_1 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.29__multipart_key1.2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB.3 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.29__shadow_key1.2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB.3_1 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.29__multipart_key1.2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB.4 2024-03-04T18:16:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.29__shadow_key1.2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB.4_1 2024-03-04T18:16:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.028+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.29__multipart_key1.2~FdFjeIBwgIJN7Cyo4_kELoBWZysT0KB.5 2024-03-04T18:16:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.036+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.036+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:16:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.8057876764085551121', time=2024-03-04T20:15:37.312649+0000, chain.objs.size()=9 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.46__multipart_key1.2~6z9EL8GynUKXEqsTgXQ9BvoxcC9_nI8.1 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.46__shadow_key1.2~6z9EL8GynUKXEqsTgXQ9BvoxcC9_nI8.1_1 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.46__multipart_key1.2~6z9EL8GynUKXEqsTgXQ9BvoxcC9_nI8.2 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.46__shadow_key1.2~6z9EL8GynUKXEqsTgXQ9BvoxcC9_nI8.2_1 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.46__multipart_key1.2~6z9EL8GynUKXEqsTgXQ9BvoxcC9_nI8.3 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.46__shadow_key1.2~6z9EL8GynUKXEqsTgXQ9BvoxcC9_nI8.3_1 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.46__multipart_key1.2~6z9EL8GynUKXEqsTgXQ9BvoxcC9_nI8.4 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.46__shadow_key1.2~6z9EL8GynUKXEqsTgXQ9BvoxcC9_nI8.4_1 2024-03-04T18:16:38.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.040+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.46__multipart_key1.2~6z9EL8GynUKXEqsTgXQ9BvoxcC9_nI8.5 2024-03-04T18:16:38.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.044+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:38.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.048+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:16:38.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.052+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.052+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.052+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.052+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:16:38.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.052+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-03-04T18:16:38.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.15218067410460605711', time=2024-03-04T20:13:18.452567+0000, chain.objs.size()=21 2024-03-04T18:16:38.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.1 2024-03-04T18:16:38.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.1_1 2024-03-04T18:16:38.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.2 2024-03-04T18:16:38.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.2_1 2024-03-04T18:16:38.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.3 2024-03-04T18:16:38.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.3_1 2024-03-04T18:16:38.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.4 2024-03-04T18:16:38.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.4_1 2024-03-04T18:16:38.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.5 2024-03-04T18:16:38.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.5_1 2024-03-04T18:16:38.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.6 2024-03-04T18:16:38.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.056+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.6_1 2024-03-04T18:16:38.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.7 2024-03-04T18:16:38.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.7_1 2024-03-04T18:16:38.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.8 2024-03-04T18:16:38.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.8_1 2024-03-04T18:16:38.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.9 2024-03-04T18:16:38.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.9_1 2024-03-04T18:16:38.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.10 2024-03-04T18:16:38.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.10_1 2024-03-04T18:16:38.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~E1yPzWDuay9LSbLJkV2OqbHq-YQbLCo.11 2024-03-04T18:16:38.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.3871095294837214784', time=2024-03-04T20:14:36.868724+0000, chain.objs.size()=16 2024-03-04T18:16:38.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.1 2024-03-04T18:16:38.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.060+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.1_1 2024-03-04T18:16:38.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.064+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.1_2 2024-03-04T18:16:38.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.068+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.2 2024-03-04T18:16:38.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.2_1 2024-03-04T18:16:38.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.2_2 2024-03-04T18:16:38.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.3 2024-03-04T18:16:38.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.3_1 2024-03-04T18:16:38.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.3_2 2024-03-04T18:16:38.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.4 2024-03-04T18:16:38.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.4_1 2024-03-04T18:16:38.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.4_2 2024-03-04T18:16:38.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.5 2024-03-04T18:16:38.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.5_1 2024-03-04T18:16:38.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.072+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.5_2 2024-03-04T18:16:38.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.076+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~n3p6mn9oH6gtoToqIe1SEmM6riJqTeU.6 2024-03-04T18:16:38.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.17833957724976838816', time=2024-03-04T20:15:05.324628+0000, chain.objs.size()=16 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.1 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.1_1 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.1_2 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.2 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.2_1 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.2_2 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.3 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.3_1 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.3_2 2024-03-04T18:16:38.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.4 2024-03-04T18:16:38.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.080+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.4_1 2024-03-04T18:16:38.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.084+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.4_2 2024-03-04T18:16:38.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.084+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.5 2024-03-04T18:16:38.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.5_1 2024-03-04T18:16:38.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__shadow_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.5_2 2024-03-04T18:16:38.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.44__multipart_key1.2~zcsUd9qSDxE938jpb2WJwKcN9P10t0Q.6 2024-03-04T18:16:38.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.4451913490112274782', time=2024-03-04T20:15:23.478904+0000, chain.objs.size()=9 2024-03-04T18:16:38.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.45__multipart_key1.2~FKj4stfcpMvs8H3VXRhOKywhfVgfod9.1 2024-03-04T18:16:38.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.45__shadow_key1.2~FKj4stfcpMvs8H3VXRhOKywhfVgfod9.1_1 2024-03-04T18:16:38.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.45__multipart_key1.2~FKj4stfcpMvs8H3VXRhOKywhfVgfod9.2 2024-03-04T18:16:38.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.45__shadow_key1.2~FKj4stfcpMvs8H3VXRhOKywhfVgfod9.2_1 2024-03-04T18:16:38.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.45__multipart_key1.2~FKj4stfcpMvs8H3VXRhOKywhfVgfod9.3 2024-03-04T18:16:38.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.088+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.45__shadow_key1.2~FKj4stfcpMvs8H3VXRhOKywhfVgfod9.3_1 2024-03-04T18:16:38.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.092+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.45__multipart_key1.2~FKj4stfcpMvs8H3VXRhOKywhfVgfod9.4 2024-03-04T18:16:38.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.092+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.45__shadow_key1.2~FKj4stfcpMvs8H3VXRhOKywhfVgfod9.4_1 2024-03-04T18:16:38.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.092+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.45__multipart_key1.2~FKj4stfcpMvs8H3VXRhOKywhfVgfod9.5 2024-03-04T18:16:38.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.096+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:38.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.100+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:16:38.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.100+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.100+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.100+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:16:38.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.100+0000 7fc099a02980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.10298034478663164698', time=2024-03-04T20:10:11.638166+0000, chain.objs.size()=7 2024-03-04T18:16:38.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.100+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~NLQ5cP9fkEZ65QX6KG42nL0Qb8yi-Pq.1 2024-03-04T18:16:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.100+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~NLQ5cP9fkEZ65QX6KG42nL0Qb8yi-Pq.1_1 2024-03-04T18:16:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.104+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~NLQ5cP9fkEZ65QX6KG42nL0Qb8yi-Pq.1_2 2024-03-04T18:16:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.104+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~NLQ5cP9fkEZ65QX6KG42nL0Qb8yi-Pq.2 2024-03-04T18:16:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.104+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~NLQ5cP9fkEZ65QX6KG42nL0Qb8yi-Pq.2_1 2024-03-04T18:16:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.104+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__shadow_key1.2~NLQ5cP9fkEZ65QX6KG42nL0Qb8yi-Pq.2_2 2024-03-04T18:16:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.104+0000 7fc099a02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.36__multipart_key1.2~NLQ5cP9fkEZ65QX6KG42nL0Qb8yi-Pq.3 2024-03-04T18:16:38.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.108+0000 7fc099a02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:16:38.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.108+0000 7fc099a02980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:16:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.112+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.112+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.112+0000 7fc099a02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.112+0000 7fbf4dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:38.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.116+0000 7fc099a02980 20 remove_watcher() i=0 2024-03-04T18:16:38.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.116+0000 7fc099a02980 2 removed watcher, disabling cache 2024-03-04T18:16:38.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.116+0000 7fc099a02980 20 remove_watcher() i=1 2024-03-04T18:16:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.120+0000 7fc099a02980 20 remove_watcher() i=2 2024-03-04T18:16:38.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.120+0000 7fc099a02980 20 remove_watcher() i=3 2024-03-04T18:16:38.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.120+0000 7fc099a02980 20 remove_watcher() i=4 2024-03-04T18:16:38.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.124+0000 7fc099a02980 20 remove_watcher() i=5 2024-03-04T18:16:38.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.124+0000 7fc099a02980 20 remove_watcher() i=6 2024-03-04T18:16:38.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.124+0000 7fc099a02980 20 remove_watcher() i=7 2024-03-04T18:16:38.139 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:16:38.205 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:16:38.206 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:16:38.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.248+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.248+0000 7f72bacb0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.248+0000 7f72bacb0980 20 realm 2024-03-04T18:16:38.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.248+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.248+0000 7f72bacb0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.248+0000 7f72bacb0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:16:38.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.248+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.252+0000 7f72bacb0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.252+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.252+0000 7f72bacb0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:38.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.252+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.252+0000 7f72bacb0980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:16:38.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.252+0000 7f72bacb0980 20 searching for the correct realm 2024-03-04T18:16:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.260+0000 7f72b7fc7640 0 -- 172.21.15.47:0/3172349322 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f72a0034950 msgr2=0x7f72a00193f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:38.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.268+0000 7f72b7fc7640 0 -- 172.21.15.47:0/3172349322 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x564c9bf4f530 msgr2=0x564c9bf519d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:38.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:16:38.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:16:38.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:16:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:16:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:16:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:16:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:16:38.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 zone default found 2024-03-04T18:16:38.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 4 Realm: () 2024-03-04T18:16:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:16:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:16:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:16:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 zonegroup default 2024-03-04T18:16:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 started sync module instance, tier type = 2024-03-04T18:16:38.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.276+0000 7f72bacb0980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:16:38.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.296+0000 7f72bacb0980 20 add_watcher() i=0 2024-03-04T18:16:38.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.296+0000 7f72bacb0980 20 add_watcher() i=1 2024-03-04T18:16:38.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.296+0000 7f72bacb0980 20 add_watcher() i=2 2024-03-04T18:16:38.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.296+0000 7f72bacb0980 20 add_watcher() i=3 2024-03-04T18:16:38.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.296+0000 7f72bacb0980 20 add_watcher() i=4 2024-03-04T18:16:38.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.296+0000 7f72bacb0980 20 add_watcher() i=5 2024-03-04T18:16:38.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.296+0000 7f72bacb0980 20 add_watcher() i=6 2024-03-04T18:16:38.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.300+0000 7f72bacb0980 20 add_watcher() i=7 2024-03-04T18:16:38.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.300+0000 7f72bacb0980 2 all 8 watchers are set, enabling cache 2024-03-04T18:16:38.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.300+0000 7f716efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:38.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.304+0000 7f72bacb0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:16:38.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.304+0000 7f72bacb0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:16:38.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.376+0000 7f716dfdb640 20 reqs_thread_entry: start 2024-03-04T18:16:38.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.376+0000 7f72bacb0980 20 init_complete bucket index max shards: 11 2024-03-04T18:16:38.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.376+0000 7f72bacb0980 20 Filter name: none 2024-03-04T18:16:38.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.376+0000 7f70df7fe640 20 reqs_thread_entry: start 2024-03-04T18:16:38.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.376+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:16:38.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.380+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.380+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.384+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.384+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:16:38.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.388+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.388+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.388+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.392+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:16:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.392+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.396+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.396+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.396+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:16:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.400+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.404+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.404+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.408+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:16:38.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.408+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.412+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.412+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.412+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:16:38.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.416+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.416+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.420+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.420+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:16:38.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.424+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.428+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.428+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.432+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:16:38.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.436+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.436+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.436+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.440+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:16:38.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.444+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.444+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.444+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.444+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:16:38.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.448+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.448+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.452+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.452+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:16:38.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.456+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.456+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.456+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.460+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:16:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.464+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.464+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.464+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.468+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:16:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.472+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.472+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.476+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.476+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:16:38.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.480+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.480+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.480+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.484+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:16:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.488+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.488+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.488+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.492+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:16:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.496+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.496+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.496+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.500+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:16:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.504+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.504+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.504+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.508+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:16:38.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.508+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.512+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.512+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.512+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:16:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.516+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.520+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.520+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.524+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:16:38.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.524+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.528+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.528+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.528+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:16:38.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.532+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.536+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.536+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.536+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:16:38.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.540+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.540+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.540+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.544+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:16:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.548+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.548+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.548+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.548+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:16:38.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.552+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.556+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.556+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.556+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:16:38.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.560+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.564+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.564+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.568+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:16:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.568+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.572+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.572+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.572+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:16:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.576+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.580+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.580+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.584+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:16:38.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.584+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.588+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.588+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.592+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:16:38.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.592+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.596+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.596+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.596+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:16:38.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.600+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.600+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.604+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.604+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:16:38.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.608+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.608+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.612+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.612+0000 7f72bacb0980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:16:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.616+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.620+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.620+0000 7f72bacb0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.624+0000 7f716efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:38.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.628+0000 7f72bacb0980 20 remove_watcher() i=0 2024-03-04T18:16:38.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.628+0000 7f72bacb0980 2 removed watcher, disabling cache 2024-03-04T18:16:38.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.632+0000 7f72bacb0980 20 remove_watcher() i=1 2024-03-04T18:16:38.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.636+0000 7f72bacb0980 20 remove_watcher() i=2 2024-03-04T18:16:38.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.636+0000 7f72bacb0980 20 remove_watcher() i=3 2024-03-04T18:16:38.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.640+0000 7f72bacb0980 20 remove_watcher() i=4 2024-03-04T18:16:38.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.644+0000 7f72bacb0980 20 remove_watcher() i=5 2024-03-04T18:16:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.644+0000 7f72bacb0980 20 remove_watcher() i=6 2024-03-04T18:16:38.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.648+0000 7f72bacb0980 20 remove_watcher() i=7 2024-03-04T18:16:38.668 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:16:38.744 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:16:38.744 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:16:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.788+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 realm 2024-03-04T18:16:38.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:16:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:16:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.792+0000 7f1b6d28a980 20 searching for the correct realm 2024-03-04T18:16:38.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.800+0000 7f1b6a5a1640 0 -- 172.21.15.47:0/2614216489 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f1b600b23f0 msgr2=0x7f1b600b2830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:38.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.804+0000 7f1b6ada2640 0 -- 172.21.15.47:0/2614216489 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f1b64008050 msgr2=0x7f1b6406d2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.812+0000 7f1b6d28a980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:16:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.812+0000 7f1b6d28a980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:16:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.812+0000 7f1b6d28a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:16:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.812+0000 7f1b6d28a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:16:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.812+0000 7f1b6d28a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:16:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.812+0000 7f1b6d28a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:16:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.812+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:16:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 zone default found 2024-03-04T18:16:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 4 Realm: () 2024-03-04T18:16:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:16:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:16:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:16:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 zonegroup default 2024-03-04T18:16:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados->read ofs=0 len=0 2024-03-04T18:16:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 started sync module instance, tier type = 2024-03-04T18:16:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.816+0000 7f1b6d28a980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:16:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.836+0000 7f1b6d28a980 20 add_watcher() i=0 2024-03-04T18:16:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.836+0000 7f1b6d28a980 20 add_watcher() i=1 2024-03-04T18:16:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.836+0000 7f1b6d28a980 20 add_watcher() i=2 2024-03-04T18:16:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.836+0000 7f1b6d28a980 20 add_watcher() i=3 2024-03-04T18:16:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.836+0000 7f1b6d28a980 20 add_watcher() i=4 2024-03-04T18:16:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.836+0000 7f1b6d28a980 20 add_watcher() i=5 2024-03-04T18:16:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.836+0000 7f1b6d28a980 20 add_watcher() i=6 2024-03-04T18:16:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.840+0000 7f1b6d28a980 20 add_watcher() i=7 2024-03-04T18:16:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.840+0000 7f1b6d28a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:16:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.840+0000 7f1a217e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.844+0000 7f1b6d28a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:16:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.844+0000 7f1b6d28a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:16:38.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.924+0000 7f199bfff640 20 reqs_thread_entry: start 2024-03-04T18:16:38.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.924+0000 7f1b6d28a980 20 init_complete bucket index max shards: 11 2024-03-04T18:16:38.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.924+0000 7f1b6d28a980 20 Filter name: none 2024-03-04T18:16:38.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.924+0000 7f1999ffb640 20 reqs_thread_entry: start 2024-03-04T18:16:38.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.924+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:16:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.928+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.928+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.932+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.932+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:16:38.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.936+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.936+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.936+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.940+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:16:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.940+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.940+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.940+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.944+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:16:38.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.944+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.948+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.948+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.948+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:16:38.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.952+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.956+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.956+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.956+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:16:38.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.960+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.960+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.960+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.964+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:16:38.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.968+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.968+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.968+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.972+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:16:38.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.976+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.976+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.980+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.980+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:16:38.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.984+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.984+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.988+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.988+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:16:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.992+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.992+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.992+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:38.996+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:16:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.000+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.000+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.004+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.004+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:16:39.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.008+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.008+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.012+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.012+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:16:39.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.016+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.020+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.020+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.024+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:16:39.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.024+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.028+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.028+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.032+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:16:39.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.032+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.036+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.036+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.036+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:16:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.040+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.040+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.044+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.044+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:16:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.048+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.048+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.052+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.052+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:16:39.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.060+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.060+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.060+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.064+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:16:39.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.068+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.068+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.068+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.072+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:16:39.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.076+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.076+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.076+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.080+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:16:39.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.084+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.084+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.084+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.088+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:16:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.088+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.092+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.092+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.096+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:16:39.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.100+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.100+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.100+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.104+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:16:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.108+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.108+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.108+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.112+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:16:39.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.116+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.120+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.120+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.124+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:16:39.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.128+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.128+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.128+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.132+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:16:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.136+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.136+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.140+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.140+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:16:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.144+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.148+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.148+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.152+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:16:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.152+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.156+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.156+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.160+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:16:39.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.164+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.164+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.164+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.168+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:16:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.172+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.172+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.172+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.176+0000 7f1b6d28a980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:16:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.180+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.180+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.184+0000 7f1b6d28a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.188+0000 7f1a217e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.192+0000 7f1b6d28a980 20 remove_watcher() i=0 2024-03-04T18:16:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.192+0000 7f1b6d28a980 2 removed watcher, disabling cache 2024-03-04T18:16:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.192+0000 7f1b6d28a980 20 remove_watcher() i=1 2024-03-04T18:16:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.196+0000 7f1b6d28a980 20 remove_watcher() i=2 2024-03-04T18:16:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.200+0000 7f1b6d28a980 20 remove_watcher() i=3 2024-03-04T18:16:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.200+0000 7f1b6d28a980 20 remove_watcher() i=4 2024-03-04T18:16:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.204+0000 7f1b6d28a980 20 remove_watcher() i=5 2024-03-04T18:16:39.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.208+0000 7f1b6d28a980 20 remove_watcher() i=6 2024-03-04T18:16:39.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.208+0000 7f1b6d28a980 20 remove_watcher() i=7 2024-03-04T18:16:39.232 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:16:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:16:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:16:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.352+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.352+0000 7f04af6c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.352+0000 7f04af6c9980 20 realm 2024-03-04T18:16:39.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.352+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.352+0000 7f04af6c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.352+0000 7f04af6c9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:16:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.352+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.356+0000 7f04af6c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.356+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.356+0000 7f04af6c9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.356+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.356+0000 7f04af6c9980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:16:39.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.356+0000 7f04af6c9980 20 searching for the correct realm 2024-03-04T18:16:39.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.364+0000 7f04ad1e1640 0 -- 172.21.15.47:0/3843917677 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f04a80d2040 msgr2=0x7f04a80c0930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:39.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.368+0000 7f04ad1e1640 0 -- 172.21.15.47:0/3843917677 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x55730882aad0 msgr2=0x55730882cf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:16:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:16:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:16:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:16:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:16:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:16:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.376+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 zone default found 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 4 Realm: () 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 zonegroup default 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:39.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 rados->read ofs=0 len=0 2024-03-04T18:16:39.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:39.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 started sync module instance, tier type = 2024-03-04T18:16:39.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.380+0000 7f04af6c9980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:16:39.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.400+0000 7f04af6c9980 20 add_watcher() i=0 2024-03-04T18:16:39.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.400+0000 7f04af6c9980 20 add_watcher() i=1 2024-03-04T18:16:39.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.400+0000 7f04af6c9980 20 add_watcher() i=2 2024-03-04T18:16:39.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.400+0000 7f04af6c9980 20 add_watcher() i=3 2024-03-04T18:16:39.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.400+0000 7f04af6c9980 20 add_watcher() i=4 2024-03-04T18:16:39.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.400+0000 7f04af6c9980 20 add_watcher() i=5 2024-03-04T18:16:39.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.400+0000 7f04af6c9980 20 add_watcher() i=6 2024-03-04T18:16:39.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.400+0000 7f04af6c9980 20 add_watcher() i=7 2024-03-04T18:16:39.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.400+0000 7f04af6c9980 2 all 8 watchers are set, enabling cache 2024-03-04T18:16:39.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.404+0000 7f0363fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:39.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.404+0000 7f04af6c9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:16:39.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.404+0000 7f04af6c9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:16:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.492+0000 7f0362fe5640 20 reqs_thread_entry: start 2024-03-04T18:16:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.492+0000 7f04af6c9980 20 init_complete bucket index max shards: 11 2024-03-04T18:16:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.492+0000 7f04af6c9980 20 Filter name: none 2024-03-04T18:16:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.492+0000 7f0360fe1640 20 reqs_thread_entry: start 2024-03-04T18:16:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.492+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:16:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.496+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.496+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.496+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.500+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:16:39.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.500+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.500+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.504+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.504+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:16:39.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.508+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.508+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.508+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.512+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:16:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.516+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.516+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.516+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.520+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:16:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.524+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.524+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.524+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.528+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:16:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.532+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.532+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.532+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.536+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:16:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.540+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.540+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.544+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.544+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:16:39.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.548+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.548+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.552+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.552+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:16:39.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.556+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.560+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.560+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.560+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:16:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.564+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.568+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.568+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.568+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:16:39.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.572+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.576+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.576+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.576+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:16:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.580+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.584+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.584+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.584+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:16:39.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.588+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.592+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.592+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.592+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:16:39.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.596+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.596+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.600+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.600+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:16:39.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.604+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.608+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.608+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.608+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:16:39.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.612+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.616+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.616+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.616+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:16:39.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.620+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.624+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.624+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.628+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:16:39.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.632+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.632+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.636+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.636+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:16:39.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.640+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.644+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.644+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.648+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:16:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.652+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.652+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.652+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.656+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:16:39.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.660+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.660+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.664+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.668+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:16:39.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.668+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.672+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.672+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.676+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:16:39.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.680+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.680+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.680+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.684+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:16:39.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.688+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.688+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.692+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.692+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:16:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.696+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.700+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.700+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.704+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:16:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.708+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.708+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.712+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.712+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:16:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.716+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.720+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.720+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.724+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:16:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.728+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.728+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.732+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.732+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:16:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.736+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.740+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.740+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.744+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:16:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.748+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.748+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.748+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.752+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:16:39.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.756+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.756+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.760+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.760+0000 7f04af6c9980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:16:39.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.764+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.768+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.768+0000 7f04af6c9980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.776+0000 7f0363fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:39.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.780+0000 7f04af6c9980 20 remove_watcher() i=0 2024-03-04T18:16:39.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.780+0000 7f04af6c9980 2 removed watcher, disabling cache 2024-03-04T18:16:39.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.784+0000 7f04af6c9980 20 remove_watcher() i=1 2024-03-04T18:16:39.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.788+0000 7f04af6c9980 20 remove_watcher() i=2 2024-03-04T18:16:39.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.788+0000 7f04af6c9980 20 remove_watcher() i=3 2024-03-04T18:16:39.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.792+0000 7f04af6c9980 20 remove_watcher() i=4 2024-03-04T18:16:39.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.796+0000 7f04af6c9980 20 remove_watcher() i=5 2024-03-04T18:16:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.800+0000 7f04af6c9980 20 remove_watcher() i=6 2024-03-04T18:16:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:39.800+0000 7f04af6c9980 20 remove_watcher() i=7 2024-03-04T18:16:39.824 DEBUG:teuthology.orchestra.run.smithi047:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2024-03-04T18:16:40.521 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :clean 2024-03-04T18:16:41.418 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :compileJava 2024-03-04T18:16:41.418 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :processResources 2024-03-04T18:16:41.418 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :classes 2024-03-04T18:16:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:41.918 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :compileTestJava 2024-03-04T18:16:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-03-04T18:16:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-03-04T18:16:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:41.920 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :processTestResources NO-SOURCE 2024-03-04T18:16:41.920 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :testClasses 2024-03-04T18:16:43.919 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:43.919 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :test 2024-03-04T18:16:43.919 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:43.919 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests STANDARD_OUT 2024-03-04T18:16:43.919 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04 18:16:43,416 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-03-04T18:16:43.919 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:43.919 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2024-03-04T18:16:43.920 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:43.920 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2024-03-04T18:16:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:44.618 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2024-03-04T18:16:44.718 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:44.718 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2024-03-04T18:16:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:45.418 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2024-03-04T18:16:45.518 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:45.518 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2024-03-04T18:16:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2024-03-04T18:16:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:46.818 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2024-03-04T18:16:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:46.918 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2024-03-04T18:16:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:47.318 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2024-03-04T18:16:48.118 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:48.118 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2024-03-04T18:16:48.218 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:48.218 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2024-03-04T18:16:48.320 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:48.320 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2024-03-04T18:16:48.718 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:48.718 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2024-03-04T18:16:50.218 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:50.218 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2024-03-04T18:16:50.418 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:50.418 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2024-03-04T18:16:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:51.218 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2024-03-04T18:16:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:51.219 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------- 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------- 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout:BUILD SUCCESSFUL in 11s 2024-03-04T18:16:51.410 INFO:teuthology.orchestra.run.smithi047.stdout:5 actionable tasks: 5 executed 2024-03-04T18:16:51.421 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:16:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:16:51.450 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:16:51.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.700+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.700+0000 7f15e1922980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:51.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.700+0000 7f15e1922980 20 realm 2024-03-04T18:16:51.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.700+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.700+0000 7f15e1922980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:51.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.700+0000 7f15e1922980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:16:51.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.700+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.704+0000 7f15e1922980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:51.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.704+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.704+0000 7f15e1922980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.704+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.704+0000 7f15e1922980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:16:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.704+0000 7f15e1922980 20 searching for the correct realm 2024-03-04T18:16:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.712+0000 7f15dec39640 0 -- 172.21.15.47:0/2277382300 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f15d4029040 msgr2=0x7f15d4061d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:51.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.716+0000 7f15df43a640 0 -- 172.21.15.47:0/2277382300 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f15d80d0040 msgr2=0x7f15d80dc940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:16:51.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:16:51.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:16:51.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:16:51.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:16:51.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:16:51.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:16:51.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:16:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 zone default found 2024-03-04T18:16:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 4 Realm: () 2024-03-04T18:16:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:16:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:16:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:16:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 zonegroup default 2024-03-04T18:16:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:51.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:51.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados->read ofs=0 len=0 2024-03-04T18:16:51.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:51.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 started sync module instance, tier type = 2024-03-04T18:16:51.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.728+0000 7f15e1922980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:16:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.748+0000 7f15e1922980 20 add_watcher() i=0 2024-03-04T18:16:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.748+0000 7f15e1922980 20 add_watcher() i=1 2024-03-04T18:16:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.748+0000 7f15e1922980 20 add_watcher() i=2 2024-03-04T18:16:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.748+0000 7f15e1922980 20 add_watcher() i=3 2024-03-04T18:16:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.748+0000 7f15e1922980 20 add_watcher() i=4 2024-03-04T18:16:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.748+0000 7f15e1922980 20 add_watcher() i=5 2024-03-04T18:16:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.748+0000 7f15e1922980 20 add_watcher() i=6 2024-03-04T18:16:51.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.752+0000 7f15e1922980 20 add_watcher() i=7 2024-03-04T18:16:51.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.752+0000 7f15e1922980 2 all 8 watchers are set, enabling cache 2024-03-04T18:16:51.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.756+0000 7f1495fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:51.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.756+0000 7f15e1922980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:16:51.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.756+0000 7f15e1922980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:16:51.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.836+0000 7f1494fe1640 20 reqs_thread_entry: start 2024-03-04T18:16:51.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.836+0000 7f15e1922980 20 init_complete bucket index max shards: 11 2024-03-04T18:16:51.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.836+0000 7f15e1922980 20 Filter name: none 2024-03-04T18:16:51.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.836+0000 7f140e7fc640 20 reqs_thread_entry: start 2024-03-04T18:16:51.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.836+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:16:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.840+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.840+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.840+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.840+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:16:51.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.844+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.848+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.848+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.852+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:16:51.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.852+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.856+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.856+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.860+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:16:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.860+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.860+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.864+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.864+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:16:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.868+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.868+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.868+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.868+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:16:51.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.872+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.876+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.876+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.880+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:16:51.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.884+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.884+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.884+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.884+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:16:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.888+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.892+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.892+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.892+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:16:51.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.896+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.896+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.900+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.900+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:16:51.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.904+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.904+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.904+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.908+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:16:51.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.912+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.912+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.916+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.920+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:16:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.920+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.924+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.924+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.928+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:16:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.932+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.932+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.932+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.936+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:16:51.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.940+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.944+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.944+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.948+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:16:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.952+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.952+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.952+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.956+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:16:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.960+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.960+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.960+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.964+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:16:51.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.968+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.968+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.968+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.976+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:16:51.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.980+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.980+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.980+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.984+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:16:51.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.984+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.988+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:51.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.988+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.992+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:16:51.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.996+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.996+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:51.996+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.000+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:16:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.004+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.004+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.008+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.008+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:16:52.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.012+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.016+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.016+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.020+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:16:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.024+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.024+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.028+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.028+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:16:52.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.032+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.036+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.036+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.040+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:16:52.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.044+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.044+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.044+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.048+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:16:52.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.056+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.056+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.056+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.060+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:16:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.064+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.064+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.068+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.068+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:16:52.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.072+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.076+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.076+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.080+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:16:52.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.084+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.084+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.084+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.088+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:16:52.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.092+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.092+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.096+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.096+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:16:52.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.100+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.104+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.104+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.104+0000 7f15e1922980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:16:52.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.108+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.112+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.112+0000 7f15e1922980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.116+0000 7f1495fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.124+0000 7f15e1922980 20 remove_watcher() i=0 2024-03-04T18:16:52.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.124+0000 7f15e1922980 2 removed watcher, disabling cache 2024-03-04T18:16:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.128+0000 7f15e1922980 20 remove_watcher() i=1 2024-03-04T18:16:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.128+0000 7f15e1922980 20 remove_watcher() i=2 2024-03-04T18:16:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.132+0000 7f15e1922980 20 remove_watcher() i=3 2024-03-04T18:16:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.132+0000 7f15e1922980 20 remove_watcher() i=4 2024-03-04T18:16:52.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.136+0000 7f15e1922980 20 remove_watcher() i=5 2024-03-04T18:16:52.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.140+0000 7f15e1922980 20 remove_watcher() i=6 2024-03-04T18:16:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.140+0000 7f15e1922980 20 remove_watcher() i=7 2024-03-04T18:16:52.164 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:16:52.235 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:16:52.236 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:16:52.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.280+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.280+0000 7f66beaf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:52.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.280+0000 7f66beaf8980 20 realm 2024-03-04T18:16:52.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.280+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.284+0000 7f66beaf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:52.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.284+0000 7f66beaf8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:16:52.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.284+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.284+0000 7f66beaf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:52.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.284+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.284+0000 7f66beaf8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:52.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.284+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.284+0000 7f66beaf8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:16:52.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.284+0000 7f66beaf8980 20 searching for the correct realm 2024-03-04T18:16:52.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.292+0000 7f66bbe0f640 0 -- 172.21.15.47:0/635531251 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f66b00d00b0 msgr2=0x7f66b00d07e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:52.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.508+0000 7f66bce11640 0 -- 172.21.15.47:0/635531251 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55b2abde2ac0 msgr2=0x55b2abde4ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:52.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.516+0000 7f66beaf8980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:16:52.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.516+0000 7f66beaf8980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:16:52.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.516+0000 7f66beaf8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:16:52.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.516+0000 7f66beaf8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:16:52.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.516+0000 7f66beaf8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:16:52.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.516+0000 7f66beaf8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:16:52.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.516+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:52.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:52.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:16:52.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 zone default found 2024-03-04T18:16:52.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 4 Realm: () 2024-03-04T18:16:52.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:16:52.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:16:52.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:16:52.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 zonegroup default 2024-03-04T18:16:52.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:52.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:52.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.520+0000 7f66beaf8980 20 rados->read ofs=0 len=0 2024-03-04T18:16:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.524+0000 7f66beaf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.524+0000 7f66beaf8980 20 started sync module instance, tier type = 2024-03-04T18:16:52.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.524+0000 7f66beaf8980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:16:52.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.548+0000 7f66beaf8980 20 add_watcher() i=0 2024-03-04T18:16:52.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.548+0000 7f66beaf8980 20 add_watcher() i=1 2024-03-04T18:16:52.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.548+0000 7f66beaf8980 20 add_watcher() i=2 2024-03-04T18:16:52.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.548+0000 7f66beaf8980 20 add_watcher() i=3 2024-03-04T18:16:52.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.548+0000 7f66beaf8980 20 add_watcher() i=4 2024-03-04T18:16:52.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.548+0000 7f66beaf8980 20 add_watcher() i=5 2024-03-04T18:16:52.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.548+0000 7f66beaf8980 20 add_watcher() i=6 2024-03-04T18:16:52.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.552+0000 7f66beaf8980 20 add_watcher() i=7 2024-03-04T18:16:52.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.552+0000 7f66beaf8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:16:52.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.556+0000 7f65727e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:52.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.556+0000 7f66beaf8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:16:52.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.556+0000 7f66beaf8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:16:52.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.640+0000 7f65717e2640 20 reqs_thread_entry: start 2024-03-04T18:16:52.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.640+0000 7f66beaf8980 20 init_complete bucket index max shards: 11 2024-03-04T18:16:52.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.640+0000 7f66beaf8980 20 Filter name: none 2024-03-04T18:16:52.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.644+0000 7f64e2ffd640 20 reqs_thread_entry: start 2024-03-04T18:16:52.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.644+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:16:52.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.644+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.648+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.648+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.652+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:16:52.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.652+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.656+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.656+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.660+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:16:52.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.660+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.664+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.664+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.668+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:16:52.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.668+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.672+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.672+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.676+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:16:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.676+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.680+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.680+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.684+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:16:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.684+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.688+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.688+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.692+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:16:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.692+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.696+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.696+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.700+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:16:52.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.700+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.704+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.704+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.708+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:16:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.712+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.712+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.712+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.716+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:16:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.720+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.720+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.720+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.724+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:16:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.728+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.728+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.728+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.732+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:16:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.736+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.740+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.740+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.740+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:16:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.744+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.748+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.748+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.752+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:16:52.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.756+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.756+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.756+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.760+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:16:52.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.764+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.764+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.764+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.768+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:16:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.772+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.772+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.776+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.776+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:16:52.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.780+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.784+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.784+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.788+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:16:52.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.792+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.792+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.792+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.796+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:16:52.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.800+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.804+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.804+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.808+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:16:52.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.812+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.812+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.812+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.816+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:16:52.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.820+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.824+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.824+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.828+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:16:52.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.832+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.832+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.836+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.840+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:16:52.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.840+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.844+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.844+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.848+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:16:52.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.852+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.852+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.852+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.856+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:16:52.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.860+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.860+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.864+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.864+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:16:52.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.868+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.872+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.872+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.876+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:16:52.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.880+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.880+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.880+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.884+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:16:52.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.888+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.892+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.892+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.896+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:16:52.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.896+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.900+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.900+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.904+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:16:52.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.908+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.908+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.908+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.912+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:16:52.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.916+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.920+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.920+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.924+0000 7f66beaf8980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:16:52.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.928+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.928+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:52.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.928+0000 7f66beaf8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.936+0000 7f65727e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.940+0000 7f66beaf8980 20 remove_watcher() i=0 2024-03-04T18:16:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.940+0000 7f66beaf8980 2 removed watcher, disabling cache 2024-03-04T18:16:52.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.944+0000 7f66beaf8980 20 remove_watcher() i=1 2024-03-04T18:16:52.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.944+0000 7f66beaf8980 20 remove_watcher() i=2 2024-03-04T18:16:52.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.948+0000 7f66beaf8980 20 remove_watcher() i=3 2024-03-04T18:16:52.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.948+0000 7f66beaf8980 20 remove_watcher() i=4 2024-03-04T18:16:52.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.952+0000 7f66beaf8980 20 remove_watcher() i=5 2024-03-04T18:16:52.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.956+0000 7f66beaf8980 20 remove_watcher() i=6 2024-03-04T18:16:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:52.960+0000 7f66beaf8980 20 remove_watcher() i=7 2024-03-04T18:16:52.980 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:16:53.056 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:16:53.056 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:16:53.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.100+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:53.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 realm 2024-03-04T18:16:53.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:53.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:16:53.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:53.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:53.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:16:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.104+0000 7f7939685980 20 searching for the correct realm 2024-03-04T18:16:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.108+0000 7f793719d640 0 -- 172.21.15.47:0/1461880067 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f79280c3040 msgr2=0x7f7928120120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:53.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.320+0000 7f793719d640 0 -- 172.21.15.47:0/1461880067 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f793002e060 msgr2=0x7f7930056090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:16:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:16:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:16:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:16:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:16:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:16:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:53.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:16:53.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 zone default found 2024-03-04T18:16:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 4 Realm: () 2024-03-04T18:16:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:16:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:16:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:16:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 zonegroup default 2024-03-04T18:16:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:53.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.540+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.544+0000 7f7939685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:53.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.544+0000 7f7939685980 20 rados->read ofs=0 len=0 2024-03-04T18:16:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.544+0000 7f7939685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.544+0000 7f7939685980 20 started sync module instance, tier type = 2024-03-04T18:16:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.544+0000 7f7939685980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:16:53.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.568+0000 7f7939685980 20 add_watcher() i=0 2024-03-04T18:16:53.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.568+0000 7f7939685980 20 add_watcher() i=1 2024-03-04T18:16:53.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.568+0000 7f7939685980 20 add_watcher() i=2 2024-03-04T18:16:53.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.568+0000 7f7939685980 20 add_watcher() i=3 2024-03-04T18:16:53.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.568+0000 7f7939685980 20 add_watcher() i=4 2024-03-04T18:16:53.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.568+0000 7f7939685980 20 add_watcher() i=5 2024-03-04T18:16:53.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.568+0000 7f7939685980 20 add_watcher() i=6 2024-03-04T18:16:53.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.568+0000 7f7939685980 20 add_watcher() i=7 2024-03-04T18:16:53.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.568+0000 7f7939685980 2 all 8 watchers are set, enabling cache 2024-03-04T18:16:53.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.572+0000 7f77edfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:53.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.576+0000 7f7939685980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:16:53.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.576+0000 7f7939685980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:16:53.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.672+0000 7f77ecfe1640 20 reqs_thread_entry: start 2024-03-04T18:16:53.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.672+0000 7f7939685980 20 init_complete bucket index max shards: 11 2024-03-04T18:16:53.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.672+0000 7f7939685980 20 Filter name: none 2024-03-04T18:16:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.672+0000 7f77667fc640 20 reqs_thread_entry: start 2024-03-04T18:16:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.672+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:16:53.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.676+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.680+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.680+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.680+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:16:53.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.684+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.684+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.684+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.688+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:16:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.692+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.692+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.692+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.696+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:16:53.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.696+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.700+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.700+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.700+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:16:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.704+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.708+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.708+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.708+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:16:53.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.712+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.716+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.716+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.720+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:16:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.724+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.724+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.728+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.728+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:16:53.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.732+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.736+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.736+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.740+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:16:53.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.744+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.744+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.744+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.748+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:16:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.752+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.752+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.756+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.756+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:16:53.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.760+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.764+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.764+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.768+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:16:53.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.772+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.772+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.772+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.776+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:16:53.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.780+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.780+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.780+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.784+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:16:53.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.788+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.792+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.792+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.796+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:16:53.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.796+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.800+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.800+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.804+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:16:53.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.804+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.808+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.808+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.808+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:16:53.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.812+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.816+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.816+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.816+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:16:53.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.820+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.824+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.824+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.828+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:16:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.832+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.832+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.832+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.836+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:16:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.840+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.840+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.840+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.844+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:16:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.848+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.852+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.852+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.856+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:16:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.860+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.860+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.864+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.864+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:16:53.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.868+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.872+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.872+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.876+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:16:53.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.880+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.880+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.880+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.884+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:16:53.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.888+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.888+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.888+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.892+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:16:53.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.896+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.900+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.900+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.904+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:16:53.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.908+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.908+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.908+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.912+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:16:53.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.916+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.916+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.920+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.920+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:16:53.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.924+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.928+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.928+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.932+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:16:53.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.936+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.936+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.936+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.940+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:16:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.944+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.944+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.948+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.948+0000 7f7939685980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:16:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.956+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.956+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.956+0000 7f7939685980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.964+0000 7f77edfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:53.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.968+0000 7f7939685980 20 remove_watcher() i=0 2024-03-04T18:16:53.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.968+0000 7f7939685980 2 removed watcher, disabling cache 2024-03-04T18:16:53.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.972+0000 7f7939685980 20 remove_watcher() i=1 2024-03-04T18:16:53.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.976+0000 7f7939685980 20 remove_watcher() i=2 2024-03-04T18:16:53.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.976+0000 7f7939685980 20 remove_watcher() i=3 2024-03-04T18:16:53.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.980+0000 7f7939685980 20 remove_watcher() i=4 2024-03-04T18:16:53.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.980+0000 7f7939685980 20 remove_watcher() i=5 2024-03-04T18:16:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.984+0000 7f7939685980 20 remove_watcher() i=6 2024-03-04T18:16:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:53.988+0000 7f7939685980 20 remove_watcher() i=7 2024-03-04T18:16:54.011 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:16:54.084 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:16:54.084 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:16:54.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.128+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:54.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 realm 2024-03-04T18:16:54.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:54.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:16:54.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:54.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:54.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:16:54.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.132+0000 7f14c747e980 20 searching for the correct realm 2024-03-04T18:16:54.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.140+0000 7f14c4795640 0 -- 172.21.15.47:0/2220185499 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f14ac0c1020 msgr2=0x7f14ac0f4900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:16:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.148+0000 7f14c4f96640 0 -- 172.21.15.47:0/2220185499 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f14b4025070 msgr2=0x7f14b4025960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:16:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:16:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:16:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:16:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:16:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:16:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:16:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:16:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:16:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 zone default found 2024-03-04T18:16:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 4 Realm: () 2024-03-04T18:16:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:16:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:16:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:16:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 zonegroup default 2024-03-04T18:16:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados->read ofs=0 len=0 2024-03-04T18:16:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:16:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 started sync module instance, tier type = 2024-03-04T18:16:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.156+0000 7f14c747e980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:16:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.176+0000 7f14c747e980 20 add_watcher() i=0 2024-03-04T18:16:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.176+0000 7f14c747e980 20 add_watcher() i=1 2024-03-04T18:16:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.176+0000 7f14c747e980 20 add_watcher() i=2 2024-03-04T18:16:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.176+0000 7f14c747e980 20 add_watcher() i=3 2024-03-04T18:16:54.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.176+0000 7f14c747e980 20 add_watcher() i=4 2024-03-04T18:16:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.176+0000 7f14c747e980 20 add_watcher() i=5 2024-03-04T18:16:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.176+0000 7f14c747e980 20 add_watcher() i=6 2024-03-04T18:16:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.180+0000 7f14c747e980 20 add_watcher() i=7 2024-03-04T18:16:54.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.180+0000 7f14c747e980 2 all 8 watchers are set, enabling cache 2024-03-04T18:16:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.184+0000 7f137afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:54.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.184+0000 7f14c747e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:16:54.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.184+0000 7f14c747e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:16:54.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.260+0000 7f1379fe3640 20 reqs_thread_entry: start 2024-03-04T18:16:54.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.260+0000 7f14c747e980 20 init_complete bucket index max shards: 11 2024-03-04T18:16:54.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.260+0000 7f14c747e980 20 Filter name: none 2024-03-04T18:16:54.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.260+0000 7f12eb7fe640 20 reqs_thread_entry: start 2024-03-04T18:16:54.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.260+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:16:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.260+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.264+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.264+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.268+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:16:54.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.268+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.272+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.272+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.276+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:16:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.276+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.276+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.276+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.280+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:16:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.284+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.284+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.284+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.288+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:16:54.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.288+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.288+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.288+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.292+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:16:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.296+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.296+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.296+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.300+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:16:54.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.304+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.308+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.308+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.312+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:16:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.312+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.312+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.316+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.316+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:16:54.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.320+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.320+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.324+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.324+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:16:54.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.328+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.332+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.332+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.336+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:16:54.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.336+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.340+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.340+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.344+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:16:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.344+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.348+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.348+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.352+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:16:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.356+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.356+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.356+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.360+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:16:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.364+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.364+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.364+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.368+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:16:54.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.372+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.372+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.376+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.376+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:16:54.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.380+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.380+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.380+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.384+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:16:54.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.388+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.388+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.388+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.392+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:16:54.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.396+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.396+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.396+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.400+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:16:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.408+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.408+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.408+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.412+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:16:54.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.416+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.416+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.416+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.420+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:16:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.424+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.424+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.424+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.428+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:16:54.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.432+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.432+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.432+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.436+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:16:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.440+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.440+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.440+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.444+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:16:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.448+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.448+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.448+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.452+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:16:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.456+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.456+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.456+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.460+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:16:54.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.464+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.464+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.464+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.468+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:16:54.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.472+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.476+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.476+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.480+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:16:54.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.484+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.484+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.484+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.488+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:16:54.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.492+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.492+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.492+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.496+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:16:54.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.500+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.500+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.504+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.504+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:16:54.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.508+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.512+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.512+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.516+0000 7f14c747e980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:16:54.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.520+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.520+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:16:54.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.520+0000 7f14c747e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:16:54.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.528+0000 7f137afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:16:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.532+0000 7f14c747e980 20 remove_watcher() i=0 2024-03-04T18:16:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.532+0000 7f14c747e980 2 removed watcher, disabling cache 2024-03-04T18:16:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.536+0000 7f14c747e980 20 remove_watcher() i=1 2024-03-04T18:16:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.540+0000 7f14c747e980 20 remove_watcher() i=2 2024-03-04T18:16:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.540+0000 7f14c747e980 20 remove_watcher() i=3 2024-03-04T18:16:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.544+0000 7f14c747e980 20 remove_watcher() i=4 2024-03-04T18:16:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.544+0000 7f14c747e980 20 remove_watcher() i=5 2024-03-04T18:16:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.548+0000 7f14c747e980 20 remove_watcher() i=6 2024-03-04T18:16:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:16:54.552+0000 7f14c747e980 20 remove_watcher() i=7 2024-03-04T18:16:54.574 DEBUG:teuthology.orchestra.run.smithi047:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2024-03-04T18:16:55.341 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :clean 2024-03-04T18:16:55.939 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :compileJava 2024-03-04T18:16:55.939 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :processResources 2024-03-04T18:16:55.939 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :classes 2024-03-04T18:16:56.338 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:56.339 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :compileTestJava 2024-03-04T18:16:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-03-04T18:16:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-03-04T18:16:56.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:56.340 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :processTestResources NO-SOURCE 2024-03-04T18:16:56.340 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :testClasses 2024-03-04T18:16:58.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:58.340 INFO:teuthology.orchestra.run.smithi047.stdout:> Task :test 2024-03-04T18:16:58.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:58.340 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests STANDARD_OUT 2024-03-04T18:16:58.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04 18:16:57,731 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-03-04T18:16:58.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:58.340 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2024-03-04T18:16:58.438 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:58.439 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2024-03-04T18:16:59.739 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:59.739 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2024-03-04T18:16:59.939 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:16:59.939 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2024-03-04T18:17:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:00.738 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2024-03-04T18:17:00.938 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:00.939 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2024-03-04T18:17:01.638 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:01.639 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2024-03-04T18:17:14.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:14.539 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2024-03-04T18:17:15.238 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:15.239 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2024-03-04T18:17:15.438 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:15.439 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2024-03-04T18:17:16.139 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:16.139 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2024-03-04T18:17:16.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2024-03-04T18:17:17.339 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:17.339 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2024-03-04T18:17:17.339 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:17.339 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2024-03-04T18:17:17.438 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:17.439 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2024-03-04T18:17:17.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:17.639 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2024-03-04T18:17:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:18.439 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2024-03-04T18:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2024-03-04T18:17:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:28.439 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2024-03-04T18:17:28.739 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:28.739 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2024-03-04T18:17:29.439 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:29.439 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2024-03-04T18:17:29.839 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:29.839 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2024-03-04T18:17:30.538 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:30.539 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2024-03-04T18:17:30.739 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:30.739 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2024-03-04T18:17:31.438 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:31.439 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2024-03-04T18:17:31.638 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:31.639 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2024-03-04T18:17:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:32.339 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2024-03-04T18:17:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:32.539 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2024-03-04T18:17:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:33.239 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2024-03-04T18:17:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:33.439 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2024-03-04T18:17:34.139 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:34.139 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2024-03-04T18:17:34.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:34.539 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2024-03-04T18:17:35.139 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:35.139 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2024-03-04T18:17:48.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:48.539 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2024-03-04T18:17:49.939 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:17:49.939 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2024-03-04T18:18:35.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:35.640 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2024-03-04T18:18:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:37.039 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2024-03-04T18:18:37.139 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:37.139 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2024-03-04T18:18:37.839 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:37.840 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2024-03-04T18:18:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:38.139 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2024-03-04T18:18:39.339 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:39.339 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2024-03-04T18:18:53.739 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:53.739 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2024-03-04T18:18:54.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:54.539 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2024-03-04T18:18:54.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:54.540 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2024-03-04T18:18:54.739 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:18:54.739 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2024-03-04T18:19:13.039 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:19:13.039 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2024-03-04T18:19:16.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:19:16.940 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests STANDARD_OUT 2024-03-04T18:19:16.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04 18:19:16,862 [s3-transfer-manager-worker-1] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:19:16.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04 18:19:16,868 [s3-transfer-manager-worker-10] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:19:16.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04 18:19:16,879 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:19:19.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:19:19.640 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2024-03-04T18:19:51.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:19:51.941 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2024-03-04T18:19:53.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:19:53.040 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2024-03-04T18:20:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:20:05.240 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2024-03-04T18:20:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:20:05.940 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2024-03-04T18:21:23.241 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:21:23.241 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2024-03-04T18:21:23.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:21:23.940 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2024-03-04T18:21:41.440 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:21:41.440 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2024-03-04T18:21:42.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:21:42.140 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2024-03-04T18:21:55.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:21:55.041 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2024-03-04T18:21:55.840 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:21:55.840 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2024-03-04T18:22:00.440 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:00.440 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2024-03-04T18:22:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:15.041 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2024-03-04T18:22:15.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:15.240 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2024-03-04T18:22:15.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:15.740 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2024-03-04T18:22:16.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:16.641 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2024-03-04T18:22:21.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:21.041 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2024-03-04T18:22:21.242 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:21.243 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2024-03-04T18:22:21.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:21.941 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2024-03-04T18:22:22.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:22.140 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2024-03-04T18:22:22.840 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:22.841 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2024-03-04T18:22:23.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:23.040 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2024-03-04T18:22:23.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:23.740 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2024-03-04T18:22:23.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:23.941 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2024-03-04T18:22:24.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:24.640 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2024-03-04T18:22:24.841 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:24.841 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2024-03-04T18:22:25.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:25.641 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2024-03-04T18:22:26.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:26.140 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2024-03-04T18:22:27.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:27.141 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2024-03-04T18:22:27.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:27.641 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2024-03-04T18:22:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:28.641 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2024-03-04T18:22:29.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:29.140 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2024-03-04T18:22:30.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:30.141 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2024-03-04T18:22:30.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:30.641 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2024-03-04T18:22:31.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:31.540 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2024-03-04T18:22:32.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:32.140 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2024-03-04T18:22:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:33.041 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2024-03-04T18:22:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:33.641 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2024-03-04T18:22:34.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:34.541 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2024-03-04T18:22:35.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:35.140 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2024-03-04T18:22:36.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:36.041 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2024-03-04T18:22:36.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:36.541 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2024-03-04T18:22:37.440 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:37.441 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2024-03-04T18:22:37.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:37.940 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2024-03-04T18:22:38.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:38.941 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2024-03-04T18:22:39.440 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:39.441 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2024-03-04T18:22:40.440 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:40.441 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2024-03-04T18:22:41.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:41.141 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2024-03-04T18:22:42.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:42.241 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2024-03-04T18:22:42.741 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:42.741 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2024-03-04T18:22:43.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:43.740 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2024-03-04T18:22:44.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:44.341 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2024-03-04T18:22:45.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:45.241 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2024-03-04T18:22:45.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:45.741 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2024-03-04T18:22:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:46.741 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2024-03-04T18:22:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:47.240 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2024-03-04T18:22:48.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:48.241 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2024-03-04T18:22:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:48.640 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2024-03-04T18:22:49.641 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:49.641 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2024-03-04T18:22:50.141 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:50.141 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2024-03-04T18:22:51.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:51.140 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2024-03-04T18:22:51.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:51.640 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2024-03-04T18:22:52.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:52.540 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2024-03-04T18:22:53.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:53.041 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2024-03-04T18:22:53.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:53.941 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2024-03-04T18:22:54.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:54.341 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2024-03-04T18:22:55.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:55.240 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2024-03-04T18:22:55.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:55.740 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2024-03-04T18:22:56.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:56.541 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2024-03-04T18:22:57.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:57.041 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2024-03-04T18:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2024-03-04T18:22:58.141 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:58.141 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2024-03-04T18:22:58.841 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:58.841 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2024-03-04T18:22:59.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:59.040 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2024-03-04T18:22:59.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:59.741 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2024-03-04T18:22:59.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:22:59.941 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2024-03-04T18:23:00.741 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:00.741 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2024-03-04T18:23:01.040 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:01.040 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2024-03-04T18:23:01.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:01.741 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2024-03-04T18:23:02.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:02.341 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2024-03-04T18:23:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:03.041 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2024-03-04T18:23:03.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:03.140 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2024-03-04T18:23:03.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:03.240 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2024-03-04T18:23:03.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:03.541 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2024-03-04T18:23:04.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2024-03-04T18:23:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:04.541 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2024-03-04T18:23:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:05.341 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2024-03-04T18:23:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:05.640 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2024-03-04T18:23:06.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:06.341 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2024-03-04T18:23:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:06.541 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2024-03-04T18:23:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:07.241 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2024-03-04T18:23:20.641 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:20.641 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2024-03-04T18:23:21.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:21.341 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testUploadFileList STARTED 2024-03-04T18:23:34.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:34.940 INFO:teuthology.orchestra.run.smithi047.stdout:suite > Object tests > ObjectTest.testUploadFileList PASSED 2024-03-04T18:23:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout:---------------------------------------------------------------------- 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout:| Results: SUCCESS (70 tests, 70 successes, 0 failures, 0 skipped) | 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout:---------------------------------------------------------------------- 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout:BUILD SUCCESSFUL in 6m 41s 2024-03-04T18:23:35.831 INFO:teuthology.orchestra.run.smithi047.stdout:5 actionable tasks: 5 executed 2024-03-04T18:23:35.842 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:23:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:35.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.909+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.909+0000 7fe38ca90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.909+0000 7fe38ca90980 20 realm 2024-03-04T18:23:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.909+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.909+0000 7fe38ca90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.909+0000 7fe38ca90980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:35.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.909+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.909+0000 7fe38ca90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:35.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.909+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.913+0000 7fe38ca90980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:35.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.913+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.913+0000 7fe38ca90980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:35.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.913+0000 7fe38ca90980 20 searching for the correct realm 2024-03-04T18:23:35.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.921+0000 7fe38a5a8640 0 -- 172.21.15.47:0/137883740 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7fe37c053040 msgr2=0x7fe37c054940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:23:35.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.925+0000 7fe38a5a8640 0 -- 172.21.15.47:0/137883740 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fe3840bf030 msgr2=0x7fe3840edae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:35.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.933+0000 7fe38ca90980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:35.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.933+0000 7fe38ca90980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:35.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.933+0000 7fe38ca90980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:35.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.933+0000 7fe38ca90980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:35.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.933+0000 7fe38ca90980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:35.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.933+0000 7fe38ca90980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:35.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.933+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:35.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:35.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 zone default found 2024-03-04T18:23:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 4 Realm: () 2024-03-04T18:23:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 zonegroup default 2024-03-04T18:23:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados->read ofs=0 len=0 2024-03-04T18:23:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 started sync module instance, tier type = 2024-03-04T18:23:35.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.937+0000 7fe38ca90980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.957+0000 7fe38ca90980 20 add_watcher() i=0 2024-03-04T18:23:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.957+0000 7fe38ca90980 20 add_watcher() i=1 2024-03-04T18:23:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.957+0000 7fe38ca90980 20 add_watcher() i=2 2024-03-04T18:23:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.957+0000 7fe38ca90980 20 add_watcher() i=3 2024-03-04T18:23:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.957+0000 7fe38ca90980 20 add_watcher() i=4 2024-03-04T18:23:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.957+0000 7fe38ca90980 20 add_watcher() i=5 2024-03-04T18:23:35.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.957+0000 7fe38ca90980 20 add_watcher() i=6 2024-03-04T18:23:35.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.961+0000 7fe38ca90980 20 add_watcher() i=7 2024-03-04T18:23:35.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.961+0000 7fe38ca90980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:35.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.961+0000 7fe240fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:35.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.965+0000 7fe38ca90980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:35.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:35.965+0000 7fe38ca90980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.041+0000 7fe1bb7fe640 20 reqs_thread_entry: start 2024-03-04T18:23:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.041+0000 7fe38ca90980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.041+0000 7fe38ca90980 20 Filter name: none 2024-03-04T18:23:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.041+0000 7fe1b97fa640 20 reqs_thread_entry: start 2024-03-04T18:23:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.041+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.045+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.045+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.13432981865420105284', time=2024-03-04T20:20:57.169502+0000, chain.objs.size()=9 2024-03-04T18:23:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~9jW1zrw36sMc_NjbbV07E8mSLnkz-XE.1 2024-03-04T18:23:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~9jW1zrw36sMc_NjbbV07E8mSLnkz-XE.1_1 2024-03-04T18:23:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~9jW1zrw36sMc_NjbbV07E8mSLnkz-XE.2 2024-03-04T18:23:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~9jW1zrw36sMc_NjbbV07E8mSLnkz-XE.2_1 2024-03-04T18:23:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~9jW1zrw36sMc_NjbbV07E8mSLnkz-XE.3 2024-03-04T18:23:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~9jW1zrw36sMc_NjbbV07E8mSLnkz-XE.3_1 2024-03-04T18:23:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~9jW1zrw36sMc_NjbbV07E8mSLnkz-XE.4 2024-03-04T18:23:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~9jW1zrw36sMc_NjbbV07E8mSLnkz-XE.4_1 2024-03-04T18:23:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.049+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~9jW1zrw36sMc_NjbbV07E8mSLnkz-XE.5 2024-03-04T18:23:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.057+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.065+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:23:36.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.065+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.069+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.069+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.069+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:23:36.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.073+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.073+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.077+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.077+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:23:36.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.081+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.081+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:23:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.2563635327725237669', time=2024-03-04T20:19:16.487323+0000, chain.objs.size()=9 2024-03-04T18:23:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.88__multipart_key1.2~al9m64wOT8E3MRJvM3jVs18q0TM7fet.1 2024-03-04T18:23:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.88__shadow_key1.2~al9m64wOT8E3MRJvM3jVs18q0TM7fet.1_1 2024-03-04T18:23:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.88__multipart_key1.2~al9m64wOT8E3MRJvM3jVs18q0TM7fet.2 2024-03-04T18:23:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.88__shadow_key1.2~al9m64wOT8E3MRJvM3jVs18q0TM7fet.2_1 2024-03-04T18:23:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.88__multipart_key1.2~al9m64wOT8E3MRJvM3jVs18q0TM7fet.3 2024-03-04T18:23:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.88__shadow_key1.2~al9m64wOT8E3MRJvM3jVs18q0TM7fet.3_1 2024-03-04T18:23:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.88__multipart_key1.2~al9m64wOT8E3MRJvM3jVs18q0TM7fet.4 2024-03-04T18:23:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.88__shadow_key1.2~al9m64wOT8E3MRJvM3jVs18q0TM7fet.4_1 2024-03-04T18:23:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.88__multipart_key1.2~al9m64wOT8E3MRJvM3jVs18q0TM7fet.5 2024-03-04T18:23:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.4736074153858880610', time=2024-03-04T20:21:23.398670+0000, chain.objs.size()=7 2024-03-04T18:23:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~k1IZFZl-Fy0n-GrvNPPVAfG3fwgKJrm.1 2024-03-04T18:23:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~k1IZFZl-Fy0n-GrvNPPVAfG3fwgKJrm.1_1 2024-03-04T18:23:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.085+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~k1IZFZl-Fy0n-GrvNPPVAfG3fwgKJrm.1_2 2024-03-04T18:23:36.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.097+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~k1IZFZl-Fy0n-GrvNPPVAfG3fwgKJrm.2 2024-03-04T18:23:36.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.097+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~k1IZFZl-Fy0n-GrvNPPVAfG3fwgKJrm.2_1 2024-03-04T18:23:36.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.097+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~k1IZFZl-Fy0n-GrvNPPVAfG3fwgKJrm.2_2 2024-03-04T18:23:36.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.097+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~k1IZFZl-Fy0n-GrvNPPVAfG3fwgKJrm.3 2024-03-04T18:23:36.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.105+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.113+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:23:36.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.113+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:23:36.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.12480625933166537651', time=2024-03-04T20:18:25.108408+0000, chain.objs.size()=9 2024-03-04T18:23:36.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~Kxf3M9Sewo0TP5aQhlv2Ftb17swZab8.1 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~Kxf3M9Sewo0TP5aQhlv2Ftb17swZab8.1_1 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~Kxf3M9Sewo0TP5aQhlv2Ftb17swZab8.2 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~Kxf3M9Sewo0TP5aQhlv2Ftb17swZab8.2_1 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~Kxf3M9Sewo0TP5aQhlv2Ftb17swZab8.3 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~Kxf3M9Sewo0TP5aQhlv2Ftb17swZab8.3_1 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~Kxf3M9Sewo0TP5aQhlv2Ftb17swZab8.4 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~Kxf3M9Sewo0TP5aQhlv2Ftb17swZab8.4_1 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~Kxf3M9Sewo0TP5aQhlv2Ftb17swZab8.5 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.5217113703452725107', time=2024-03-04T20:18:35.815201+0000, chain.objs.size()=5 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.82__shadow_.gDmcQOYw7e2W6xBDcgltpQohyrO6MCa_1 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.82__shadow_.gDmcQOYw7e2W6xBDcgltpQohyrO6MCa_2 2024-03-04T18:23:36.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.117+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.82__shadow_.gDmcQOYw7e2W6xBDcgltpQohyrO6MCa_3 2024-03-04T18:23:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.125+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.82__shadow_.gDmcQOYw7e2W6xBDcgltpQohyrO6MCa_4 2024-03-04T18:23:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.125+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.82__shadow_.gDmcQOYw7e2W6xBDcgltpQohyrO6MCa_5 2024-03-04T18:23:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.133+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.141+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:23:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.141+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.141+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.141+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.145+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:23:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.145+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.149+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.149+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.149+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:23:36.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:23:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.15488376775742004387', time=2024-03-04T20:17:27.843977+0000, chain.objs.size()=2 2024-03-04T18:23:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.72__shadow_.-8jRVCpfJyp8YW3CyZopDdWjG2370O5_1 2024-03-04T18:23:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.72__shadow_.-8jRVCpfJyp8YW3CyZopDdWjG2370O5_2 2024-03-04T18:23:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.16217987769752959549', time=2024-03-04T20:17:49.339775+0000, chain.objs.size()=9 2024-03-04T18:23:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.1 2024-03-04T18:23:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__shadow_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.1_1 2024-03-04T18:23:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.2 2024-03-04T18:23:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__shadow_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.2_1 2024-03-04T18:23:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.3 2024-03-04T18:23:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__shadow_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.3_1 2024-03-04T18:23:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.4 2024-03-04T18:23:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__shadow_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.4_1 2024-03-04T18:23:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.153+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.5 2024-03-04T18:23:36.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.161+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.169+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:23:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.15329750016228599880', time=2024-03-04T20:21:10.082119+0000, chain.objs.size()=7 2024-03-04T18:23:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~r1YmhVhRe6M1JA8YRcCo6omXWCm3_6V.1 2024-03-04T18:23:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~r1YmhVhRe6M1JA8YRcCo6omXWCm3_6V.1_1 2024-03-04T18:23:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~r1YmhVhRe6M1JA8YRcCo6omXWCm3_6V.1_2 2024-03-04T18:23:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~r1YmhVhRe6M1JA8YRcCo6omXWCm3_6V.2 2024-03-04T18:23:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~r1YmhVhRe6M1JA8YRcCo6omXWCm3_6V.2_1 2024-03-04T18:23:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~r1YmhVhRe6M1JA8YRcCo6omXWCm3_6V.2_2 2024-03-04T18:23:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.173+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~r1YmhVhRe6M1JA8YRcCo6omXWCm3_6V.3 2024-03-04T18:23:36.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.181+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.189+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:23:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.193+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.193+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.197+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.197+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.10484352399667020864', time=2024-03-04T20:18:36.455472+0000, chain.objs.size()=7 2024-03-04T18:23:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.197+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~6gUpSuyBk7Kuu0GLvKDpLZGR7Z9IoF2.1 2024-03-04T18:23:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.197+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~6gUpSuyBk7Kuu0GLvKDpLZGR7Z9IoF2.1_1 2024-03-04T18:23:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.197+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~6gUpSuyBk7Kuu0GLvKDpLZGR7Z9IoF2.1_2 2024-03-04T18:23:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.197+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~6gUpSuyBk7Kuu0GLvKDpLZGR7Z9IoF2.2 2024-03-04T18:23:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.197+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~6gUpSuyBk7Kuu0GLvKDpLZGR7Z9IoF2.2_1 2024-03-04T18:23:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.197+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~6gUpSuyBk7Kuu0GLvKDpLZGR7Z9IoF2.2_2 2024-03-04T18:23:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.197+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~6gUpSuyBk7Kuu0GLvKDpLZGR7Z9IoF2.3 2024-03-04T18:23:36.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.205+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.209+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:23:36.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.209+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.209+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.213+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.213+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:23:36.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.4841425977683554660', time=2024-03-04T20:23:20.784679+0000, chain.objs.size()=9 2024-03-04T18:23:36.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.132__multipart_key1.2~ZH4JBZhKljOEZusmBLpWk40kDJLKqw0.1 2024-03-04T18:23:36.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.132__shadow_key1.2~ZH4JBZhKljOEZusmBLpWk40kDJLKqw0.1_1 2024-03-04T18:23:36.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.132__multipart_key1.2~ZH4JBZhKljOEZusmBLpWk40kDJLKqw0.2 2024-03-04T18:23:36.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.132__shadow_key1.2~ZH4JBZhKljOEZusmBLpWk40kDJLKqw0.2_1 2024-03-04T18:23:36.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.132__multipart_key1.2~ZH4JBZhKljOEZusmBLpWk40kDJLKqw0.3 2024-03-04T18:23:36.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.132__shadow_key1.2~ZH4JBZhKljOEZusmBLpWk40kDJLKqw0.3_1 2024-03-04T18:23:36.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.132__multipart_key1.2~ZH4JBZhKljOEZusmBLpWk40kDJLKqw0.4 2024-03-04T18:23:36.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.132__shadow_key1.2~ZH4JBZhKljOEZusmBLpWk40kDJLKqw0.4_1 2024-03-04T18:23:36.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.217+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.132__multipart_key1.2~ZH4JBZhKljOEZusmBLpWk40kDJLKqw0.5 2024-03-04T18:23:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.225+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.233+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:23:36.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.233+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.237+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.237+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.237+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:23:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.14400888679537495977', time=2024-03-04T20:18:35.506868+0000, chain.objs.size()=7 2024-03-04T18:23:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~Q1Y0FkwbHAsQjKlpP7TXjzsYM_1O0VN.1 2024-03-04T18:23:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~Q1Y0FkwbHAsQjKlpP7TXjzsYM_1O0VN.1_1 2024-03-04T18:23:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~Q1Y0FkwbHAsQjKlpP7TXjzsYM_1O0VN.1_2 2024-03-04T18:23:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~Q1Y0FkwbHAsQjKlpP7TXjzsYM_1O0VN.2 2024-03-04T18:23:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~Q1Y0FkwbHAsQjKlpP7TXjzsYM_1O0VN.2_1 2024-03-04T18:23:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~Q1Y0FkwbHAsQjKlpP7TXjzsYM_1O0VN.2_2 2024-03-04T18:23:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.241+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~Q1Y0FkwbHAsQjKlpP7TXjzsYM_1O0VN.3 2024-03-04T18:23:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.249+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.253+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:23:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.257+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.257+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.257+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.257+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:23:36.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.261+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.261+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.261+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.265+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:23:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.12954953480831629569', time=2024-03-04T20:18:30.168318+0000, chain.objs.size()=7 2024-03-04T18:23:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~aJTaH73lC4Bg-eygGTVFQEy3e3jTjmk.1 2024-03-04T18:23:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~aJTaH73lC4Bg-eygGTVFQEy3e3jTjmk.1_1 2024-03-04T18:23:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~aJTaH73lC4Bg-eygGTVFQEy3e3jTjmk.1_2 2024-03-04T18:23:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~aJTaH73lC4Bg-eygGTVFQEy3e3jTjmk.2 2024-03-04T18:23:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~aJTaH73lC4Bg-eygGTVFQEy3e3jTjmk.2_1 2024-03-04T18:23:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~aJTaH73lC4Bg-eygGTVFQEy3e3jTjmk.2_2 2024-03-04T18:23:36.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.269+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~aJTaH73lC4Bg-eygGTVFQEy3e3jTjmk.3 2024-03-04T18:23:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.277+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.281+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:23:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.281+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.285+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.285+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.285+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:23:36.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.289+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.289+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.289+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.289+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:23:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.5818257848781643851', time=2024-03-04T20:21:41.578539+0000, chain.objs.size()=9 2024-03-04T18:23:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.92__multipart_key1.2~sGrafN7EgWVmGkW4QSTc7yX26AZqhfZ.1 2024-03-04T18:23:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.92__shadow_key1.2~sGrafN7EgWVmGkW4QSTc7yX26AZqhfZ.1_1 2024-03-04T18:23:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.92__multipart_key1.2~sGrafN7EgWVmGkW4QSTc7yX26AZqhfZ.2 2024-03-04T18:23:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.92__shadow_key1.2~sGrafN7EgWVmGkW4QSTc7yX26AZqhfZ.2_1 2024-03-04T18:23:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.92__multipart_key1.2~sGrafN7EgWVmGkW4QSTc7yX26AZqhfZ.3 2024-03-04T18:23:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.92__shadow_key1.2~sGrafN7EgWVmGkW4QSTc7yX26AZqhfZ.3_1 2024-03-04T18:23:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.92__multipart_key1.2~sGrafN7EgWVmGkW4QSTc7yX26AZqhfZ.4 2024-03-04T18:23:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.92__shadow_key1.2~sGrafN7EgWVmGkW4QSTc7yX26AZqhfZ.4_1 2024-03-04T18:23:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.293+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.92__multipart_key1.2~sGrafN7EgWVmGkW4QSTc7yX26AZqhfZ.5 2024-03-04T18:23:36.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.301+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.305+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:23:36.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.17192342819055929483', time=2024-03-04T20:20:31.423204+0000, chain.objs.size()=9 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~grD-dFkgAJHe9rQBfs6N7zNXPUYnVB_.1 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~grD-dFkgAJHe9rQBfs6N7zNXPUYnVB_.1_1 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~grD-dFkgAJHe9rQBfs6N7zNXPUYnVB_.2 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~grD-dFkgAJHe9rQBfs6N7zNXPUYnVB_.2_1 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~grD-dFkgAJHe9rQBfs6N7zNXPUYnVB_.3 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~grD-dFkgAJHe9rQBfs6N7zNXPUYnVB_.3_1 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~grD-dFkgAJHe9rQBfs6N7zNXPUYnVB_.4 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~grD-dFkgAJHe9rQBfs6N7zNXPUYnVB_.4_1 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~grD-dFkgAJHe9rQBfs6N7zNXPUYnVB_.5 2024-03-04T18:23:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.14482700357167597714', time=2024-03-04T20:20:44.161290+0000, chain.objs.size()=9 2024-03-04T18:23:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~8lsIceG9_x2ovt7Y80xWm3oH_icVm-C.1 2024-03-04T18:23:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~8lsIceG9_x2ovt7Y80xWm3oH_icVm-C.1_1 2024-03-04T18:23:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.309+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~8lsIceG9_x2ovt7Y80xWm3oH_icVm-C.2 2024-03-04T18:23:36.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.317+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~8lsIceG9_x2ovt7Y80xWm3oH_icVm-C.2_1 2024-03-04T18:23:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.317+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~8lsIceG9_x2ovt7Y80xWm3oH_icVm-C.3 2024-03-04T18:23:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.317+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~8lsIceG9_x2ovt7Y80xWm3oH_icVm-C.3_1 2024-03-04T18:23:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.317+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~8lsIceG9_x2ovt7Y80xWm3oH_icVm-C.4 2024-03-04T18:23:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.317+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~8lsIceG9_x2ovt7Y80xWm3oH_icVm-C.4_1 2024-03-04T18:23:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.317+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~8lsIceG9_x2ovt7Y80xWm3oH_icVm-C.5 2024-03-04T18:23:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.317+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.14179915429168709342', time=2024-03-04T20:23:35.144091+0000, chain.objs.size()=9 2024-03-04T18:23:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.317+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.133__multipart_key1/data/file.mpg.2~2lZ9Fd0jTk3qL-rg39SL9Db3EG0a0XY.1 2024-03-04T18:23:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.317+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.133__shadow_key1/data/file.mpg.2~2lZ9Fd0jTk3qL-rg39SL9Db3EG0a0XY.1_1 2024-03-04T18:23:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.321+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.133__multipart_key1/data/file.mpg.2~2lZ9Fd0jTk3qL-rg39SL9Db3EG0a0XY.2 2024-03-04T18:23:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.321+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.133__shadow_key1/data/file.mpg.2~2lZ9Fd0jTk3qL-rg39SL9Db3EG0a0XY.2_1 2024-03-04T18:23:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.321+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.133__multipart_key1/data/file.mpg.2~2lZ9Fd0jTk3qL-rg39SL9Db3EG0a0XY.3 2024-03-04T18:23:36.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.325+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.133__shadow_key1/data/file.mpg.2~2lZ9Fd0jTk3qL-rg39SL9Db3EG0a0XY.3_1 2024-03-04T18:23:36.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.325+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.133__multipart_key1/data/file.mpg.2~2lZ9Fd0jTk3qL-rg39SL9Db3EG0a0XY.4 2024-03-04T18:23:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.329+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.133__shadow_key1/data/file.mpg.2~2lZ9Fd0jTk3qL-rg39SL9Db3EG0a0XY.4_1 2024-03-04T18:23:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.329+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.133__multipart_key1/data/file.mpg.2~2lZ9Fd0jTk3qL-rg39SL9Db3EG0a0XY.5 2024-03-04T18:23:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.337+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.341+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:23:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.341+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.341+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.341+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.345+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:23:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.345+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.345+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.4630625933033872296', time=2024-03-04T20:19:52.140569+0000, chain.objs.size()=9 2024-03-04T18:23:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file.mpg.2~rdWZFaxvZmO17tXnC5TV_SlgkLUS_z8.1 2024-03-04T18:23:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__shadow_file.mpg.2~rdWZFaxvZmO17tXnC5TV_SlgkLUS_z8.1_1 2024-03-04T18:23:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file.mpg.2~rdWZFaxvZmO17tXnC5TV_SlgkLUS_z8.2 2024-03-04T18:23:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__shadow_file.mpg.2~rdWZFaxvZmO17tXnC5TV_SlgkLUS_z8.2_1 2024-03-04T18:23:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file.mpg.2~rdWZFaxvZmO17tXnC5TV_SlgkLUS_z8.3 2024-03-04T18:23:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__shadow_file.mpg.2~rdWZFaxvZmO17tXnC5TV_SlgkLUS_z8.3_1 2024-03-04T18:23:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file.mpg.2~rdWZFaxvZmO17tXnC5TV_SlgkLUS_z8.4 2024-03-04T18:23:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__shadow_file.mpg.2~rdWZFaxvZmO17tXnC5TV_SlgkLUS_z8.4_1 2024-03-04T18:23:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.349+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file.mpg.2~rdWZFaxvZmO17tXnC5TV_SlgkLUS_z8.5 2024-03-04T18:23:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.353+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.357+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:23:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:23:36.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.80__shadow_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.1_1_fe4e40e8dac60f48773d5d71c0ee2fd0', time=2024-03-04T20:17:48.717358+0000, chain.objs.size()=9 2024-03-04T18:23:36.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.1 2024-03-04T18:23:36.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__shadow_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.1_1 2024-03-04T18:23:36.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.2 2024-03-04T18:23:36.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__shadow_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.2_1 2024-03-04T18:23:36.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.3 2024-03-04T18:23:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__shadow_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.3_1 2024-03-04T18:23:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.4 2024-03-04T18:23:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__shadow_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.4_1 2024-03-04T18:23:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.80__multipart_key1.2~YWUSsHVqvGZlJBtMFnYtJpGuHDBKGtC.5 2024-03-04T18:23:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.15428293879214262693', time=2024-03-04T20:19:52.292969+0000, chain.objs.size()=5 2024-03-04T18:23:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file2.mpg.2~XVhwSSBAkBuI72N8x8MRj869LqZ0YMt.1 2024-03-04T18:23:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file2.mpg.2~XVhwSSBAkBuI72N8x8MRj869LqZ0YMt.2 2024-03-04T18:23:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.361+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file2.mpg.2~XVhwSSBAkBuI72N8x8MRj869LqZ0YMt.3 2024-03-04T18:23:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file2.mpg.2~XVhwSSBAkBuI72N8x8MRj869LqZ0YMt.4 2024-03-04T18:23:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.89__multipart_file2.mpg.2~XVhwSSBAkBuI72N8x8MRj869LqZ0YMt.5 2024-03-04T18:23:36.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.9439326509944718887', time=2024-03-04T20:21:55.231928+0000, chain.objs.size()=9 2024-03-04T18:23:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.93__multipart_key1.2~TEC0bMmCYBf-uSqQPBtlZXvtMWibRRe.1 2024-03-04T18:23:36.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.93__shadow_key1.2~TEC0bMmCYBf-uSqQPBtlZXvtMWibRRe.1_1 2024-03-04T18:23:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.93__multipart_key1.2~TEC0bMmCYBf-uSqQPBtlZXvtMWibRRe.2 2024-03-04T18:23:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.93__shadow_key1.2~TEC0bMmCYBf-uSqQPBtlZXvtMWibRRe.2_1 2024-03-04T18:23:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.93__multipart_key1.2~TEC0bMmCYBf-uSqQPBtlZXvtMWibRRe.3 2024-03-04T18:23:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.93__shadow_key1.2~TEC0bMmCYBf-uSqQPBtlZXvtMWibRRe.3_1 2024-03-04T18:23:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.93__multipart_key1.2~TEC0bMmCYBf-uSqQPBtlZXvtMWibRRe.4 2024-03-04T18:23:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.365+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.93__shadow_key1.2~TEC0bMmCYBf-uSqQPBtlZXvtMWibRRe.4_1 2024-03-04T18:23:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.369+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.93__multipart_key1.2~TEC0bMmCYBf-uSqQPBtlZXvtMWibRRe.5 2024-03-04T18:23:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.377+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.381+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:23:36.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.381+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.5144724186208232727', time=2024-03-04T20:18:14.426453+0000, chain.objs.size()=9 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~SiB86tS8gc5Lq1TxIX3Y6sxlfpMd28_.1 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~SiB86tS8gc5Lq1TxIX3Y6sxlfpMd28_.1_1 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~SiB86tS8gc5Lq1TxIX3Y6sxlfpMd28_.2 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~SiB86tS8gc5Lq1TxIX3Y6sxlfpMd28_.2_1 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~SiB86tS8gc5Lq1TxIX3Y6sxlfpMd28_.3 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~SiB86tS8gc5Lq1TxIX3Y6sxlfpMd28_.3_1 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~SiB86tS8gc5Lq1TxIX3Y6sxlfpMd28_.4 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~SiB86tS8gc5Lq1TxIX3Y6sxlfpMd28_.4_1 2024-03-04T18:23:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.385+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~SiB86tS8gc5Lq1TxIX3Y6sxlfpMd28_.5 2024-03-04T18:23:36.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.389+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.393+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:23:36.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.397+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.397+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.397+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.397+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:23:36.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.397+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.9942762369681767559', time=2024-03-04T20:21:23.100706+0000, chain.objs.size()=7 2024-03-04T18:23:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~q3vCmTGPulxSInz1r2vEXmV9p7cxRF3.1 2024-03-04T18:23:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~q3vCmTGPulxSInz1r2vEXmV9p7cxRF3.1_1 2024-03-04T18:23:36.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~q3vCmTGPulxSInz1r2vEXmV9p7cxRF3.1_2 2024-03-04T18:23:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~q3vCmTGPulxSInz1r2vEXmV9p7cxRF3.2 2024-03-04T18:23:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~q3vCmTGPulxSInz1r2vEXmV9p7cxRF3.2_1 2024-03-04T18:23:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__shadow_key1.2~q3vCmTGPulxSInz1r2vEXmV9p7cxRF3.2_2 2024-03-04T18:23:36.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.401+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.91__multipart_key1.2~q3vCmTGPulxSInz1r2vEXmV9p7cxRF3.3 2024-03-04T18:23:36.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.405+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:23:36.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:23:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.16653753710498884150', time=2024-03-04T20:18:20.445724+0000, chain.objs.size()=9 2024-03-04T18:23:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~j2F0O-wo4swJLIl3zrhneVMVrytYC9Q.1 2024-03-04T18:23:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~j2F0O-wo4swJLIl3zrhneVMVrytYC9Q.1_1 2024-03-04T18:23:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~j2F0O-wo4swJLIl3zrhneVMVrytYC9Q.2 2024-03-04T18:23:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~j2F0O-wo4swJLIl3zrhneVMVrytYC9Q.2_1 2024-03-04T18:23:36.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~j2F0O-wo4swJLIl3zrhneVMVrytYC9Q.3 2024-03-04T18:23:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~j2F0O-wo4swJLIl3zrhneVMVrytYC9Q.3_1 2024-03-04T18:23:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~j2F0O-wo4swJLIl3zrhneVMVrytYC9Q.4 2024-03-04T18:23:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__shadow_key1.2~j2F0O-wo4swJLIl3zrhneVMVrytYC9Q.4_1 2024-03-04T18:23:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.83__multipart_key1.2~j2F0O-wo4swJLIl3zrhneVMVrytYC9Q.5 2024-03-04T18:23:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='797b67a4-b876-4d04-b122-10afd858d465.4453.10400454655129243650', time=2024-03-04T20:18:53.886266+0000, chain.objs.size()=9 2024-03-04T18:23:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.87__multipart_key1.2~ocETTDx4Fn652GI_TOFUsy5X5xyWOTN.1 2024-03-04T18:23:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.409+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.87__shadow_key1.2~ocETTDx4Fn652GI_TOFUsy5X5xyWOTN.1_1 2024-03-04T18:23:36.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.413+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.87__multipart_key1.2~ocETTDx4Fn652GI_TOFUsy5X5xyWOTN.2 2024-03-04T18:23:36.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.417+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.87__shadow_key1.2~ocETTDx4Fn652GI_TOFUsy5X5xyWOTN.2_1 2024-03-04T18:23:36.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.417+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.87__multipart_key1.2~ocETTDx4Fn652GI_TOFUsy5X5xyWOTN.3 2024-03-04T18:23:36.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.417+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.87__shadow_key1.2~ocETTDx4Fn652GI_TOFUsy5X5xyWOTN.3_1 2024-03-04T18:23:36.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.417+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.87__multipart_key1.2~ocETTDx4Fn652GI_TOFUsy5X5xyWOTN.4 2024-03-04T18:23:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.417+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.87__shadow_key1.2~ocETTDx4Fn652GI_TOFUsy5X5xyWOTN.4_1 2024-03-04T18:23:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.417+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.87__multipart_key1.2~ocETTDx4Fn652GI_TOFUsy5X5xyWOTN.5 2024-03-04T18:23:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.425+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.425+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:23:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.429+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.429+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.429+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.429+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:23:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-03-04T18:23:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS', time=2024-03-04T20:22:09.882934+0000, chain.objs.size()=1 2024-03-04T18:23:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.15 2024-03-04T18:23:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS', time=2024-03-04T20:22:10.059811+0000, chain.objs.size()=1 2024-03-04T18:23:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.16 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS', time=2024-03-04T20:22:10.326974+0000, chain.objs.size()=1 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.17 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS', time=2024-03-04T20:22:10.440114+0000, chain.objs.size()=1 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.19 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS', time=2024-03-04T20:22:10.846911+0000, chain.objs.size()=1 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.20 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS', time=2024-03-04T20:22:11.045704+0000, chain.objs.size()=39 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.1 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.2 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.3 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.4 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.5 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.6 2024-03-04T18:23:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.433+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.7 2024-03-04T18:23:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.8 2024-03-04T18:23:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.9 2024-03-04T18:23:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.10 2024-03-04T18:23:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.11 2024-03-04T18:23:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.12 2024-03-04T18:23:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.13 2024-03-04T18:23:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.14 2024-03-04T18:23:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.ix-QL3yZjErh5pf7fWBrMs772amV5L-c.15 2024-03-04T18:23:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.Gz8Sj2PlqwiDfc8FSvM3GNTPJg8t6quD.16 2024-03-04T18:23:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.d4r5fa9qgzh2QUS5X5acZZVzHbtgvdxH.17 2024-03-04T18:23:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.441+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.18 2024-03-04T18:23:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.lWUKxeIHaUXbyqGYV8zU8iZ3R6e7_rCj.19 2024-03-04T18:23:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.4QP08-cszapWm-ud6ECnDVXI2bpxW2EV.20 2024-03-04T18:23:36.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.21 2024-03-04T18:23:36.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.22 2024-03-04T18:23:36.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.23 2024-03-04T18:23:36.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.24 2024-03-04T18:23:36.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.25 2024-03-04T18:23:36.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.26 2024-03-04T18:23:36.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.27 2024-03-04T18:23:36.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.28 2024-03-04T18:23:36.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.445+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.29 2024-03-04T18:23:36.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.449+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.30 2024-03-04T18:23:36.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.449+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.31 2024-03-04T18:23:36.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.453+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.32 2024-03-04T18:23:36.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.453+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.33 2024-03-04T18:23:36.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.453+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.34 2024-03-04T18:23:36.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.453+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.35 2024-03-04T18:23:36.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.453+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.36 2024-03-04T18:23:36.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.453+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.37 2024-03-04T18:23:36.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.453+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.38 2024-03-04T18:23:36.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.453+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.94__multipart_key1.2~h_mEvYb2UUzYTLO6vnWYMqJ12vbsGRS.39 2024-03-04T18:23:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.457+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.461+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:23:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:23:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 20 garbage collection: RGWGC::process iterating over entry tag='2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv', time=2024-03-04T20:17:14.457518+0000, chain.objs.size()=9 2024-03-04T18:23:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.68__multipart_key1.2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv.1 2024-03-04T18:23:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.68__shadow_key1.2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv.1_1 2024-03-04T18:23:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.68__multipart_key1.2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv.2 2024-03-04T18:23:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.68__shadow_key1.2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv.2_1 2024-03-04T18:23:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.68__multipart_key1.2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv.3 2024-03-04T18:23:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.68__shadow_key1.2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv.3_1 2024-03-04T18:23:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.68__multipart_key1.2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv.4 2024-03-04T18:23:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.68__shadow_key1.2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv.4_1 2024-03-04T18:23:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.465+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:797b67a4-b876-4d04-b122-10afd858d465.4453.68__multipart_key1.2~Bzx5cUrTEVwJXO7F-MrK4ebEnFlx7Jv.5 2024-03-04T18:23:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.469+0000 7fe38ca90980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:23:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.473+0000 7fe38ca90980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:23:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.473+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.473+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.477+0000 7fe38ca90980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.477+0000 7fe240fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.481+0000 7fe38ca90980 20 remove_watcher() i=0 2024-03-04T18:23:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.481+0000 7fe38ca90980 2 removed watcher, disabling cache 2024-03-04T18:23:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.481+0000 7fe38ca90980 20 remove_watcher() i=1 2024-03-04T18:23:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.481+0000 7fe38ca90980 20 remove_watcher() i=2 2024-03-04T18:23:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.481+0000 7fe38ca90980 20 remove_watcher() i=3 2024-03-04T18:23:36.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.485+0000 7fe38ca90980 20 remove_watcher() i=4 2024-03-04T18:23:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.485+0000 7fe38ca90980 20 remove_watcher() i=5 2024-03-04T18:23:36.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.485+0000 7fe38ca90980 20 remove_watcher() i=6 2024-03-04T18:23:36.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.489+0000 7fe38ca90980 20 remove_watcher() i=7 2024-03-04T18:23:36.504 DEBUG:teuthology.orchestra.run.smithi047:> radosgw-admin gc process --include-all 2024-03-04T18:23:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:36.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.609+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:36.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 20 realm 2024-03-04T18:23:36.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:36.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.613+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.617+0000 7fc2f9fac980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.617+0000 7fc2f9fac980 20 searching for the correct realm 2024-03-04T18:23:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.621+0000 7fc2f72c3640 0 -- 172.21.15.47:0/4220583706 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fc2e006e400 msgr2=0x7fc2e007b920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.629+0000 7fc2f7ac4640 0 -- 172.21.15.47:0/4220583706 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fc2f0027040 msgr2=0x7fc2f000e450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:23:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:36.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 zone default found 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 4 Realm: () 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 zonegroup default 2024-03-04T18:23:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.637+0000 7fc2f9fac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.641+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.641+0000 7fc2f9fac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:36.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.641+0000 7fc2f9fac980 20 rados->read ofs=0 len=0 2024-03-04T18:23:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.641+0000 7fc2f9fac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.641+0000 7fc2f9fac980 20 started sync module instance, tier type = 2024-03-04T18:23:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.641+0000 7fc2f9fac980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.657+0000 7fc2f9fac980 20 add_watcher() i=0 2024-03-04T18:23:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.657+0000 7fc2f9fac980 20 add_watcher() i=1 2024-03-04T18:23:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.657+0000 7fc2f9fac980 20 add_watcher() i=2 2024-03-04T18:23:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.657+0000 7fc2f9fac980 20 add_watcher() i=3 2024-03-04T18:23:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.657+0000 7fc2f9fac980 20 add_watcher() i=4 2024-03-04T18:23:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.657+0000 7fc2f9fac980 20 add_watcher() i=5 2024-03-04T18:23:36.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.657+0000 7fc2f9fac980 20 add_watcher() i=6 2024-03-04T18:23:36.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.661+0000 7fc2f9fac980 20 add_watcher() i=7 2024-03-04T18:23:36.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.661+0000 7fc2f9fac980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.665+0000 7fc1adfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.665+0000 7fc2f9fac980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.665+0000 7fc2f9fac980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.745+0000 7fc1acfe1640 20 reqs_thread_entry: start 2024-03-04T18:23:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.745+0000 7fc2f9fac980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.745+0000 7fc2f9fac980 20 Filter name: none 2024-03-04T18:23:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.745+0000 7fc1267fc640 20 reqs_thread_entry: start 2024-03-04T18:23:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.745+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:23:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.749+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.753+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.753+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.757+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:23:36.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.761+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.761+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.761+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.765+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:23:36.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.769+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.769+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.769+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.773+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:23:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.777+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.777+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.777+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.781+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:23:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.785+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.785+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.789+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.789+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:23:36.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.793+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.797+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.797+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.797+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:23:36.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.801+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.801+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.805+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.805+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:23:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.809+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.809+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.809+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.813+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:23:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.817+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.817+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.817+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.821+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:23:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.821+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.825+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.825+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.825+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:23:36.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.829+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.829+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.829+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.833+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:23:36.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.833+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.837+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.837+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.837+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:23:36.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.841+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.841+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.841+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.845+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:23:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.845+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.849+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.849+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.853+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:23:36.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.853+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.857+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.857+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.857+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:23:36.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.861+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.861+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.865+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.865+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:23:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.869+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.873+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.873+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.877+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:23:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.877+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.880+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.880+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.880+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:23:36.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.884+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.884+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.884+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.888+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:23:36.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.888+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.892+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.892+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.896+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:23:36.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.896+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.896+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.896+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.900+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:23:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.904+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.904+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.904+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.908+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:23:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.912+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.912+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.912+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.916+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:23:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.916+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.920+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.920+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.920+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:23:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.924+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.924+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.924+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.928+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:23:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.932+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.932+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.932+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.932+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:23:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.936+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.940+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.940+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.944+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:23:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.944+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.948+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.948+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.948+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:23:36.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.952+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.956+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.956+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.960+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:23:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.960+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.964+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.964+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.964+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:23:36.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.968+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.972+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.972+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.976+0000 7fc2f9fac980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:23:36.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.976+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.980+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:23:36.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.980+0000 7fc2f9fac980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:23:36.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.984+0000 7fc1adfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:36.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.988+0000 7fc2f9fac980 20 remove_watcher() i=0 2024-03-04T18:23:36.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.988+0000 7fc2f9fac980 2 removed watcher, disabling cache 2024-03-04T18:23:36.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.992+0000 7fc2f9fac980 20 remove_watcher() i=1 2024-03-04T18:23:36.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.992+0000 7fc2f9fac980 20 remove_watcher() i=2 2024-03-04T18:23:37.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:36.996+0000 7fc2f9fac980 20 remove_watcher() i=3 2024-03-04T18:23:37.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:37.000+0000 7fc2f9fac980 20 remove_watcher() i=4 2024-03-04T18:23:37.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:37.000+0000 7fc2f9fac980 20 remove_watcher() i=5 2024-03-04T18:23:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:37.004+0000 7fc2f9fac980 20 remove_watcher() i=6 2024-03-04T18:23:37.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:37.008+0000 7fc2f9fac980 20 remove_watcher() i=7 2024-03-04T18:23:37.033 INFO:teuthology.run_tasks:Running task s3tests... 2024-03-04T18:23:37.052 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}} 2024-03-04T18:23:37.052 INFO:tasks.s3tests:Downloading s3-tests... 2024-03-04T18:23:37.052 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2024-03-04T18:23:37.052 DEBUG:teuthology.orchestra.run.smithi047:> git clone -b ceph-master https://git.ceph.com/s3-tests.git /home/ubuntu/cephtest/s3-tests-client.0 2024-03-04T18:23:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-client.0'... 2024-03-04T18:23:37.826 INFO:tasks.s3tests:Creating rgw users... 2024-03-04T18:23:37.826 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2024-03-04T18:23:37.826 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --email foo.client.0+test@test.test --caps 'user-policy=*' --access-key DDFPEZWEBBYHMFSSPIII --secret DCcCyA2YbLgGHYeHDxQ/t/OeLoGAuVfTKNIdOvJid/FHsdCMMKKTNg== --cluster ceph 2024-03-04T18:23:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:38.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.156+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 realm 2024-03-04T18:23:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.160+0000 7f50a2ec9980 20 searching for the correct realm 2024-03-04T18:23:38.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.168+0000 7f50a01e0640 0 -- 172.21.15.47:0/4164235824 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f50941168b0 msgr2=0x7f5094128040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:23:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.384+0000 7f50a09e1640 0 -- 172.21.15.47:0/4164235824 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f5090058d90 msgr2=0x7f5090057a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:23:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:38.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:38.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.604+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 zone default found 2024-03-04T18:23:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 4 Realm: () 2024-03-04T18:23:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 zonegroup default 2024-03-04T18:23:38.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 started sync module instance, tier type = 2024-03-04T18:23:38.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.608+0000 7f50a2ec9980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:38.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.632+0000 7f50a2ec9980 20 add_watcher() i=0 2024-03-04T18:23:38.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.632+0000 7f50a2ec9980 20 add_watcher() i=1 2024-03-04T18:23:38.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.632+0000 7f50a2ec9980 20 add_watcher() i=2 2024-03-04T18:23:38.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.632+0000 7f50a2ec9980 20 add_watcher() i=3 2024-03-04T18:23:38.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.632+0000 7f50a2ec9980 20 add_watcher() i=4 2024-03-04T18:23:38.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.632+0000 7f50a2ec9980 20 add_watcher() i=5 2024-03-04T18:23:38.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.632+0000 7f50a2ec9980 20 add_watcher() i=6 2024-03-04T18:23:38.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.636+0000 7f50a2ec9980 20 add_watcher() i=7 2024-03-04T18:23:38.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.636+0000 7f50a2ec9980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:38.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f4f56fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:38.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:38.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:38.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 5 note: GC not initialized 2024-03-04T18:23:38.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f4f55fe3640 20 reqs_thread_entry: start 2024-03-04T18:23:38.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:38.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 20 Filter name: none 2024-03-04T18:23:38.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f4ec77fe640 20 reqs_thread_entry: start 2024-03-04T18:23:38.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-03-04T18:23:38.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2024-03-04T18:23:38.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:23:38.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2024-03-04T18:23:38.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.640+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 10 cache get: name=default.rgw.meta+users.keys+DDFPEZWEBBYHMFSSPIII : miss 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 10 cache put: name=default.rgw.meta+users.keys+DDFPEZWEBBYHMFSSPIII info.flags=0x0 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 10 adding default.rgw.meta+users.keys+DDFPEZWEBBYHMFSSPIII to cache LRU end 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 10 cache get: name=default.rgw.meta+users.keys+DDFPEZWEBBYHMFSSPIII : hit (negative entry) 2024-03-04T18:23:38.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.644+0000 7f50a2ec9980 10 cache get: name=default.rgw.meta+users.keys+DDFPEZWEBBYHMFSSPIII : hit (negative entry) 2024-03-04T18:23:38.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.648+0000 7f50a2ec9980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-03-04T18:23:38.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.648+0000 7f50a2ec9980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:23:38.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.648+0000 7f50a2ec9980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-03-04T18:23:38.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.648+0000 7f4f77fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595150 cookie 94530503230624 notifier 7599 bl.length()=641 2024-03-04T18:23:38.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.648+0000 7f4f77fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-03-04T18:23:38.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.648+0000 7f4f77fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:23:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.652+0000 7f50a2ec9980 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2024-03-04T18:23:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.652+0000 7f50a2ec9980 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2024-03-04T18:23:38.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.652+0000 7f50a2ec9980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.email:foo.client.0+test@test.test, ofs0, ns] 2024-03-04T18:23:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.652+0000 7f4f777fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595274 cookie 94530503249888 notifier 7599 bl.length()=194 2024-03-04T18:23:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.652+0000 7f4f777fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2024-03-04T18:23:38.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.652+0000 7f4f777fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2024-03-04T18:23:38.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.656+0000 7f50a2ec9980 10 cache put: name=default.rgw.meta+users.keys+DDFPEZWEBBYHMFSSPIII info.flags=0x7 2024-03-04T18:23:38.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.656+0000 7f50a2ec9980 10 moving default.rgw.meta+users.keys+DDFPEZWEBBYHMFSSPIII to cache LRU end 2024-03-04T18:23:38.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.656+0000 7f50a2ec9980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:DDFPEZWEBBYHMFSSPIII, ofs0, ns] 2024-03-04T18:23:38.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.660+0000 7f4f777fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595174 cookie 94530503228416 notifier 7599 bl.length()=186 2024-03-04T18:23:38.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.660+0000 7f4f777fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+DDFPEZWEBBYHMFSSPIII info.flags=0x7 2024-03-04T18:23:38.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.660+0000 7f4f777fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+DDFPEZWEBBYHMFSSPIII to cache LRU end 2024-03-04T18:23:38.665 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-04T18:23:38.665 INFO:teuthology.orchestra.run.smithi047.stdout: "user_id": "foo.client.0", 2024-03-04T18:23:38.665 INFO:teuthology.orchestra.run.smithi047.stdout: "display_name": "Mr. foo.client.0", 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "email": "foo.client.0+test@test.test", 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "suspended": 0, 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "max_buckets": 1000, 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "subusers": [], 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "keys": [ 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "user": "foo.client.0", 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "access_key": "DDFPEZWEBBYHMFSSPIII", 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "secret_key": "DCcCyA2YbLgGHYeHDxQ/t/OeLoGAuVfTKNIdOvJid/FHsdCMMKKTNg==", 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "active": true 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "swift_keys": [], 2024-03-04T18:23:38.666 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": [ 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "user-policy", 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "perm": "*" 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "default_placement": "", 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "default_storage_class": "", 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "placement_tags": [], 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "bucket_quota": { 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:38.667 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "user_quota": { 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "temp_url_keys": [], 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "rgw", 2024-03-04T18:23:38.668 INFO:teuthology.orchestra.run.smithi047.stdout: "mfa_ids": [] 2024-03-04T18:23:38.669 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-04T18:23:38.669 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:38.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.660+0000 7f4f56fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:38.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.668+0000 7f50a2ec9980 20 remove_watcher() i=0 2024-03-04T18:23:38.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.668+0000 7f50a2ec9980 2 removed watcher, disabling cache 2024-03-04T18:23:38.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.668+0000 7f50a2ec9980 20 remove_watcher() i=1 2024-03-04T18:23:38.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.672+0000 7f50a2ec9980 20 remove_watcher() i=2 2024-03-04T18:23:38.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.672+0000 7f50a2ec9980 20 remove_watcher() i=3 2024-03-04T18:23:38.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.676+0000 7f50a2ec9980 20 remove_watcher() i=4 2024-03-04T18:23:38.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.680+0000 7f50a2ec9980 20 remove_watcher() i=5 2024-03-04T18:23:38.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.680+0000 7f50a2ec9980 20 remove_watcher() i=6 2024-03-04T18:23:38.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.684+0000 7f50a2ec9980 20 remove_watcher() i=7 2024-03-04T18:23:38.705 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 9564509505 --totp-seed YHQUTEQMLZWDNLDGHYXXKAMDB7AI6XFTJOTMQTFZO23LZ6QJEKXRL3DSQFGOAMPH --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-03-04T18:23:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:38.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.780+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 20 realm 2024-03-04T18:23:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.784+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.788+0000 7fef7ad22980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.788+0000 7fef7ad22980 20 searching for the correct realm 2024-03-04T18:23:38.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.792+0000 7fef78039640 0 -- 172.21.15.47:0/3251969990 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7fef60036400 msgr2=0x7fef60036880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:38.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.800+0000 7fef78039640 0 -- 172.21.15.47:0/3251969990 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x55d22e44c880 msgr2=0x55d22e44ed20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 zone default found 2024-03-04T18:23:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 4 Realm: () 2024-03-04T18:23:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 zonegroup default 2024-03-04T18:23:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.808+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.812+0000 7fef7ad22980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.812+0000 7fef7ad22980 20 started sync module instance, tier type = 2024-03-04T18:23:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.812+0000 7fef7ad22980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.828+0000 7fef7ad22980 20 add_watcher() i=0 2024-03-04T18:23:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.828+0000 7fef7ad22980 20 add_watcher() i=1 2024-03-04T18:23:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.828+0000 7fef7ad22980 20 add_watcher() i=2 2024-03-04T18:23:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.828+0000 7fef7ad22980 20 add_watcher() i=3 2024-03-04T18:23:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.828+0000 7fef7ad22980 20 add_watcher() i=4 2024-03-04T18:23:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.828+0000 7fef7ad22980 20 add_watcher() i=5 2024-03-04T18:23:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.828+0000 7fef7ad22980 20 add_watcher() i=6 2024-03-04T18:23:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.832+0000 7fef7ad22980 20 add_watcher() i=7 2024-03-04T18:23:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.832+0000 7fef7ad22980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fee2efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 5 note: GC not initialized 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fee2dfe3640 20 reqs_thread_entry: start 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 20 Filter name: none 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fed9f7fe640 20 reqs_thread_entry: start 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-03-04T18:23:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 20 rados->read ofs=0 len=0 2024-03-04T18:23:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 20 rados_obj.operate() r=0 bl.length=456 2024-03-04T18:23:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x13 2024-03-04T18:23:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:23:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:38.836+0000 7fef7ad22980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:23:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.672+0000 7fef7ad22980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-03-04T18:23:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.672+0000 7fef7ad22980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:23:40.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.672+0000 7fef7ad22980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-03-04T18:23:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.676+0000 7fee4f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529743 cookie 94361207760976 notifier 7148 bl.length()=655 2024-03-04T18:23:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.676+0000 7fee4f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-03-04T18:23:40.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.676+0000 7fee4f7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:23:40.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.680+0000 7fee2efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:40.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.684+0000 7fef7ad22980 20 remove_watcher() i=0 2024-03-04T18:23:40.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.684+0000 7fef7ad22980 2 removed watcher, disabling cache 2024-03-04T18:23:40.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.688+0000 7fef7ad22980 20 remove_watcher() i=1 2024-03-04T18:23:40.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.688+0000 7fef7ad22980 20 remove_watcher() i=2 2024-03-04T18:23:40.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.692+0000 7fef7ad22980 20 remove_watcher() i=3 2024-03-04T18:23:40.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.696+0000 7fef7ad22980 20 remove_watcher() i=4 2024-03-04T18:23:40.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.696+0000 7fef7ad22980 20 remove_watcher() i=5 2024-03-04T18:23:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.700+0000 7fef7ad22980 20 remove_watcher() i=6 2024-03-04T18:23:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.704+0000 7fef7ad22980 20 remove_watcher() i=7 2024-03-04T18:23:40.729 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2024-03-04T18:23:40.729 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name 'Mr. bar.client.0' --email bar.client.0+test@test.test --caps 'user-policy=*' --access-key ERIZTJDZISSUMDEFGXHT --secret 0mF3i2P6ZSQoPLuArNQqhbzbXKx/wSVNNlfb29QSGOHiKO0SG5T+Vw== --cluster ceph 2024-03-04T18:23:40.820 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:40.820 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:40.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.856+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 20 realm 2024-03-04T18:23:40.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:40.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.860+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.864+0000 7f7edb537980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:40.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.864+0000 7f7edb537980 20 searching for the correct realm 2024-03-04T18:23:40.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.868+0000 7f7ed904f640 0 -- 172.21.15.47:0/472938793 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f7ecc026070 msgr2=0x7f7ecc05af50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:40.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.876+0000 7f7ed904f640 0 -- 172.21.15.47:0/472938793 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f7ed40e1940 msgr2=0x7f7ed40c6420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:40.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:40.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:40.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:40.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:40.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:40.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:40.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:40.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:40.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 zone default found 2024-03-04T18:23:40.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 4 Realm: () 2024-03-04T18:23:40.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:40.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:40.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:40.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 zonegroup default 2024-03-04T18:23:40.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 started sync module instance, tier type = 2024-03-04T18:23:40.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.884+0000 7f7edb537980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.904+0000 7f7edb537980 20 add_watcher() i=0 2024-03-04T18:23:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.904+0000 7f7edb537980 20 add_watcher() i=1 2024-03-04T18:23:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.904+0000 7f7edb537980 20 add_watcher() i=2 2024-03-04T18:23:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.904+0000 7f7edb537980 20 add_watcher() i=3 2024-03-04T18:23:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.904+0000 7f7edb537980 20 add_watcher() i=4 2024-03-04T18:23:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.904+0000 7f7edb537980 20 add_watcher() i=5 2024-03-04T18:23:40.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.904+0000 7f7edb537980 20 add_watcher() i=6 2024-03-04T18:23:40.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.908+0000 7f7edb537980 20 add_watcher() i=7 2024-03-04T18:23:40.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.908+0000 7f7edb537980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7d8f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 5 note: GC not initialized 2024-03-04T18:23:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7d8e7e4640 20 reqs_thread_entry: start 2024-03-04T18:23:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 Filter name: none 2024-03-04T18:23:40.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7cfffff640 20 reqs_thread_entry: start 2024-03-04T18:23:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-03-04T18:23:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2024-03-04T18:23:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:23:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2024-03-04T18:23:40.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2024-03-04T18:23:40.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2024-03-04T18:23:40.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 cache get: name=default.rgw.meta+users.keys+ERIZTJDZISSUMDEFGXHT : miss 2024-03-04T18:23:40.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 rados->read ofs=0 len=0 2024-03-04T18:23:40.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:40.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 cache put: name=default.rgw.meta+users.keys+ERIZTJDZISSUMDEFGXHT info.flags=0x0 2024-03-04T18:23:40.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 adding default.rgw.meta+users.keys+ERIZTJDZISSUMDEFGXHT to cache LRU end 2024-03-04T18:23:40.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 cache get: name=default.rgw.meta+users.keys+ERIZTJDZISSUMDEFGXHT : hit (negative entry) 2024-03-04T18:23:40.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.912+0000 7f7edb537980 10 cache get: name=default.rgw.meta+users.keys+ERIZTJDZISSUMDEFGXHT : hit (negative entry) 2024-03-04T18:23:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.916+0000 7f7edb537980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-03-04T18:23:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.916+0000 7f7edb537980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:23:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.916+0000 7f7edb537980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-03-04T18:23:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.916+0000 7f7dbcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529912 cookie 94313141103472 notifier 7163 bl.length()=641 2024-03-04T18:23:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.916+0000 7f7dbcff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-03-04T18:23:40.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.916+0000 7f7dbcff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:23:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.916+0000 7f7edb537980 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2024-03-04T18:23:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.916+0000 7f7edb537980 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2024-03-04T18:23:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.916+0000 7f7edb537980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:bar.client.0+test@test.test, ofs0, ns] 2024-03-04T18:23:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.920+0000 7f7dbcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529913 cookie 94313141103472 notifier 7163 bl.length()=194 2024-03-04T18:23:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.920+0000 7f7dbcff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2024-03-04T18:23:40.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.920+0000 7f7dbcff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2024-03-04T18:23:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.924+0000 7f7edb537980 10 cache put: name=default.rgw.meta+users.keys+ERIZTJDZISSUMDEFGXHT info.flags=0x7 2024-03-04T18:23:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.924+0000 7f7edb537980 10 moving default.rgw.meta+users.keys+ERIZTJDZISSUMDEFGXHT to cache LRU end 2024-03-04T18:23:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.924+0000 7f7edb537980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:ERIZTJDZISSUMDEFGXHT, ofs0, ns] 2024-03-04T18:23:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.924+0000 7f7dabfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529914 cookie 94313141109776 notifier 7163 bl.length()=186 2024-03-04T18:23:40.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.924+0000 7f7dabfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+ERIZTJDZISSUMDEFGXHT info.flags=0x7 2024-03-04T18:23:40.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.924+0000 7f7dabfff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+ERIZTJDZISSUMDEFGXHT to cache LRU end 2024-03-04T18:23:40.929 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "user_id": "bar.client.0", 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "display_name": "Mr. bar.client.0", 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "email": "bar.client.0+test@test.test", 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "suspended": 0, 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "max_buckets": 1000, 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "subusers": [], 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "keys": [ 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "user": "bar.client.0", 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "access_key": "ERIZTJDZISSUMDEFGXHT", 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "secret_key": "0mF3i2P6ZSQoPLuArNQqhbzbXKx/wSVNNlfb29QSGOHiKO0SG5T+Vw==", 2024-03-04T18:23:40.930 INFO:teuthology.orchestra.run.smithi047.stdout: "active": true 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: "swift_keys": [], 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": [ 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "user-policy", 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: "perm": "*" 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: "default_placement": "", 2024-03-04T18:23:40.931 INFO:teuthology.orchestra.run.smithi047.stdout: "default_storage_class": "", 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "placement_tags": [], 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "bucket_quota": { 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "user_quota": { 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:40.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:40.933 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:40.933 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:40.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:40.933 INFO:teuthology.orchestra.run.smithi047.stdout: "temp_url_keys": [], 2024-03-04T18:23:40.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "rgw", 2024-03-04T18:23:40.933 INFO:teuthology.orchestra.run.smithi047.stdout: "mfa_ids": [] 2024-03-04T18:23:40.933 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-04T18:23:40.933 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:40.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.928+0000 7f7d8f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:40.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.932+0000 7f7edb537980 20 remove_watcher() i=0 2024-03-04T18:23:40.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.932+0000 7f7edb537980 2 removed watcher, disabling cache 2024-03-04T18:23:40.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.932+0000 7f7edb537980 20 remove_watcher() i=1 2024-03-04T18:23:40.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.936+0000 7f7edb537980 20 remove_watcher() i=2 2024-03-04T18:23:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.936+0000 7f7edb537980 20 remove_watcher() i=3 2024-03-04T18:23:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.940+0000 7f7edb537980 20 remove_watcher() i=4 2024-03-04T18:23:40.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.940+0000 7f7edb537980 20 remove_watcher() i=5 2024-03-04T18:23:40.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.944+0000 7f7edb537980 20 remove_watcher() i=6 2024-03-04T18:23:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:40.948+0000 7f7edb537980 20 remove_watcher() i=7 2024-03-04T18:23:40.966 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 3791693667 --totp-seed KWY66CUNUBAGW2NAC4L77A6HHBJ4GSUUUWKFJGW4447SKEZUAXPI4J5GE6AJMQNB --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-03-04T18:23:41.007 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:41.007 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.044+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.044+0000 7f3650bf3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.044+0000 7f3650bf3980 20 realm 2024-03-04T18:23:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.044+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.044+0000 7f3650bf3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.044+0000 7f3650bf3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.044+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.044+0000 7f3650bf3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.044+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.048+0000 7f3650bf3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.048+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.048+0000 7f3650bf3980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.048+0000 7f3650bf3980 20 searching for the correct realm 2024-03-04T18:23:41.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.056+0000 7f364e70b640 0 -- 172.21.15.47:0/3540389475 >> [v2:172.21.15.47:6800/1071303395,v1:172.21.15.47:6801/1071303395] conn(0x7f3640008030 msgr2=0x7f364004a470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:23:41.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.060+0000 7f364e70b640 0 -- 172.21.15.47:0/3540389475 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f36480a9c20 msgr2=0x7f36481201d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:41.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:41.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:41.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 zone default found 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 4 Realm: () 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 zonegroup default 2024-03-04T18:23:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.068+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.072+0000 7f3650bf3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.072+0000 7f3650bf3980 20 started sync module instance, tier type = 2024-03-04T18:23:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.072+0000 7f3650bf3980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:41.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.088+0000 7f3650bf3980 20 add_watcher() i=0 2024-03-04T18:23:41.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.088+0000 7f3650bf3980 20 add_watcher() i=1 2024-03-04T18:23:41.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.088+0000 7f3650bf3980 20 add_watcher() i=2 2024-03-04T18:23:41.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.088+0000 7f3650bf3980 20 add_watcher() i=3 2024-03-04T18:23:41.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.088+0000 7f3650bf3980 20 add_watcher() i=4 2024-03-04T18:23:41.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.088+0000 7f3650bf3980 20 add_watcher() i=5 2024-03-04T18:23:41.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.088+0000 7f3650bf3980 20 add_watcher() i=6 2024-03-04T18:23:41.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.092+0000 7f3650bf3980 20 add_watcher() i=7 2024-03-04T18:23:41.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.092+0000 7f3650bf3980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3504fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 5 note: GC not initialized 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f347f7fe640 20 reqs_thread_entry: start 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 20 Filter name: none 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f347d7fa640 20 reqs_thread_entry: start 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-03-04T18:23:41.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 20 rados_obj.operate() r=0 bl.length=456 2024-03-04T18:23:41.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x13 2024-03-04T18:23:41.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:23:41.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.096+0000 7f3650bf3980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:23:41.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.100+0000 7f3650bf3980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-03-04T18:23:41.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.100+0000 7f3650bf3980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:23:41.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.100+0000 7f3650bf3980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-03-04T18:23:41.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.100+0000 7f3531ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529915 cookie 94360783748528 notifier 7656 bl.length()=655 2024-03-04T18:23:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.100+0000 7f3531ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-03-04T18:23:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.100+0000 7f3531ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:23:41.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.104+0000 7f3504fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:41.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.108+0000 7f3650bf3980 20 remove_watcher() i=0 2024-03-04T18:23:41.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.108+0000 7f3650bf3980 2 removed watcher, disabling cache 2024-03-04T18:23:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.108+0000 7f3650bf3980 20 remove_watcher() i=1 2024-03-04T18:23:41.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.112+0000 7f3650bf3980 20 remove_watcher() i=2 2024-03-04T18:23:41.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.112+0000 7f3650bf3980 20 remove_watcher() i=3 2024-03-04T18:23:41.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.116+0000 7f3650bf3980 20 remove_watcher() i=4 2024-03-04T18:23:41.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.116+0000 7f3650bf3980 20 remove_watcher() i=5 2024-03-04T18:23:41.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.120+0000 7f3650bf3980 20 remove_watcher() i=6 2024-03-04T18:23:41.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.124+0000 7f3650bf3980 20 remove_watcher() i=7 2024-03-04T18:23:41.141 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2024-03-04T18:23:41.141 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr. testx$tenanteduser.client.0' --email 'testx$tenanteduser.client.0+test@test.test' --caps 'user-policy=*' --access-key ONMXLGXHXFNXOZJKAIAT --secret ElnqNo18Ud6z9PcsM4oB9y7xIaJ3WzA5MfHxEaaxRSoNaMSApq/Lug== --cluster ceph 2024-03-04T18:23:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:41.226 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.264+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.264+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.264+0000 7f2b3b49f980 20 realm 2024-03-04T18:23:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.264+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.264+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.264+0000 7f2b3b49f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.264+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.264+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.264+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.268+0000 7f2b3b49f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:41.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.268+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.268+0000 7f2b3b49f980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:41.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.268+0000 7f2b3b49f980 20 searching for the correct realm 2024-03-04T18:23:41.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.276+0000 7f2b387b6640 0 -- 172.21.15.47:0/3314836213 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f2b200d5070 msgr2=0x7f2b200ec420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.280+0000 7f2b397b8640 0 -- 172.21.15.47:0/3314836213 >> [v2:172.21.15.152:6800/4241326532,v1:172.21.15.152:6801/4241326532] conn(0x5577a12cd050 msgr2=0x5577a12cf510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 zone default found 2024-03-04T18:23:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 4 Realm: () 2024-03-04T18:23:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 zonegroup default 2024-03-04T18:23:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 started sync module instance, tier type = 2024-03-04T18:23:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.288+0000 7f2b3b49f980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:41.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.308+0000 7f2b3b49f980 20 add_watcher() i=0 2024-03-04T18:23:41.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.308+0000 7f2b3b49f980 20 add_watcher() i=1 2024-03-04T18:23:41.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.308+0000 7f2b3b49f980 20 add_watcher() i=2 2024-03-04T18:23:41.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.308+0000 7f2b3b49f980 20 add_watcher() i=3 2024-03-04T18:23:41.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.308+0000 7f2b3b49f980 20 add_watcher() i=4 2024-03-04T18:23:41.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.308+0000 7f2b3b49f980 20 add_watcher() i=5 2024-03-04T18:23:41.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.308+0000 7f2b3b49f980 20 add_watcher() i=6 2024-03-04T18:23:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.312+0000 7f2b3b49f980 20 add_watcher() i=7 2024-03-04T18:23:41.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.312+0000 7f2b3b49f980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:41.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f29ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:41.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:41.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 5 note: GC not initialized 2024-03-04T18:23:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f29ee7e4640 20 reqs_thread_entry: start 2024-03-04T18:23:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 20 Filter name: none 2024-03-04T18:23:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f29e7fff640 20 reqs_thread_entry: start 2024-03-04T18:23:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-03-04T18:23:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2024-03-04T18:23:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:23:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test : miss 2024-03-04T18:23:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x0 2024-03-04T18:23:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 10 adding default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2024-03-04T18:23:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 10 cache get: name=default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT : miss 2024-03-04T18:23:41.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.316+0000 7f2b3b49f980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.320+0000 7f2b3b49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.320+0000 7f2b3b49f980 10 cache put: name=default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT info.flags=0x0 2024-03-04T18:23:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.320+0000 7f2b3b49f980 10 adding default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT to cache LRU end 2024-03-04T18:23:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.320+0000 7f2b3b49f980 10 cache get: name=default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT : hit (negative entry) 2024-03-04T18:23:41.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.320+0000 7f2b3b49f980 10 cache get: name=default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT : hit (negative entry) 2024-03-04T18:23:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.324+0000 7f2b3b49f980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-03-04T18:23:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.324+0000 7f2b3b49f980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:23:41.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.324+0000 7f2b3b49f980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-03-04T18:23:41.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.324+0000 7f2a0ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529867 cookie 93972293602384 notifier 7196 bl.length()=715 2024-03-04T18:23:41.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.324+0000 7f2a0ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-03-04T18:23:41.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.324+0000 7f2a0ffff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:23:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.328+0000 7f2b3b49f980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2024-03-04T18:23:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.328+0000 7f2b3b49f980 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2024-03-04T18:23:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.328+0000 7f2b3b49f980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.email:testx$tenanteduser.client.0+test@test.test, ofs0, ns] 2024-03-04T18:23:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.328+0000 7f2a0ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529868 cookie 93972293607888 notifier 7196 bl.length()=224 2024-03-04T18:23:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.328+0000 7f2a0ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2024-03-04T18:23:41.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.328+0000 7f2a0ffff640 10 rgw watcher librados: moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2024-03-04T18:23:41.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.336+0000 7f2b3b49f980 10 cache put: name=default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT info.flags=0x7 2024-03-04T18:23:41.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.336+0000 7f2b3b49f980 10 moving default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT to cache LRU end 2024-03-04T18:23:41.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.336+0000 7f2b3b49f980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:ONMXLGXHXFNXOZJKAIAT, ofs0, ns] 2024-03-04T18:23:41.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.336+0000 7f2a0ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529773 cookie 93972293594192 notifier 7196 bl.length()=201 2024-03-04T18:23:41.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.336+0000 7f2a0ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT info.flags=0x7 2024-03-04T18:23:41.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.336+0000 7f2a0ffff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT to cache LRU end 2024-03-04T18:23:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.340+0000 7f2a0f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529774 cookie 93972293594192 notifier 7196 bl.length()=201 2024-03-04T18:23:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.340+0000 7f2a0f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT info.flags=0x7 2024-03-04T18:23:41.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.340+0000 7f2a0f7fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+ONMXLGXHXFNXOZJKAIAT to cache LRU end 2024-03-04T18:23:41.348 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-04T18:23:41.348 INFO:teuthology.orchestra.run.smithi047.stdout: "user_id": "testx$tenanteduser.client.0", 2024-03-04T18:23:41.348 INFO:teuthology.orchestra.run.smithi047.stdout: "display_name": "Mr. testx$tenanteduser.client.0", 2024-03-04T18:23:41.348 INFO:teuthology.orchestra.run.smithi047.stdout: "email": "testx$tenanteduser.client.0+test@test.test", 2024-03-04T18:23:41.348 INFO:teuthology.orchestra.run.smithi047.stdout: "suspended": 0, 2024-03-04T18:23:41.348 INFO:teuthology.orchestra.run.smithi047.stdout: "max_buckets": 1000, 2024-03-04T18:23:41.348 INFO:teuthology.orchestra.run.smithi047.stdout: "subusers": [], 2024-03-04T18:23:41.348 INFO:teuthology.orchestra.run.smithi047.stdout: "keys": [ 2024-03-04T18:23:41.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: "user": "testx$tenanteduser.client.0", 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: "access_key": "ONMXLGXHXFNXOZJKAIAT", 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: "secret_key": "ElnqNo18Ud6z9PcsM4oB9y7xIaJ3WzA5MfHxEaaxRSoNaMSApq/Lug==", 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: "active": true 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: "swift_keys": [], 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": [ 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "user-policy", 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: "perm": "*" 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:41.349 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "default_placement": "", 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "default_storage_class": "", 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "placement_tags": [], 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "bucket_quota": { 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "user_quota": { 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stdout: "temp_url_keys": [], 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "rgw", 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stdout: "mfa_ids": [] 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:41.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.344+0000 7f29ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.348+0000 7f2b3b49f980 20 remove_watcher() i=0 2024-03-04T18:23:41.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.348+0000 7f2b3b49f980 2 removed watcher, disabling cache 2024-03-04T18:23:41.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.352+0000 7f2b3b49f980 20 remove_watcher() i=1 2024-03-04T18:23:41.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.352+0000 7f2b3b49f980 20 remove_watcher() i=2 2024-03-04T18:23:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.356+0000 7f2b3b49f980 20 remove_watcher() i=3 2024-03-04T18:23:41.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.356+0000 7f2b3b49f980 20 remove_watcher() i=4 2024-03-04T18:23:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.360+0000 7f2b3b49f980 20 remove_watcher() i=5 2024-03-04T18:23:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.364+0000 7f2b3b49f980 20 remove_watcher() i=6 2024-03-04T18:23:41.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.364+0000 7f2b3b49f980 20 remove_watcher() i=7 2024-03-04T18:23:41.385 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 2368198436 --totp-seed WDAKYGWRWZ2HMPB5FXEXKE2AIFSENM4SBYDA6UTBNVY2VGEUXNT7YRLB2ONBQFM6 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-03-04T18:23:41.471 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:41.471 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:41.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.508+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.508+0000 7f5d7dde6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.508+0000 7f5d7dde6980 20 realm 2024-03-04T18:23:41.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.508+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.512+0000 7f5d7dde6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.512+0000 7f5d7dde6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.512+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.512+0000 7f5d7dde6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.512+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.512+0000 7f5d7dde6980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:41.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.512+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.512+0000 7f5d7dde6980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:41.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.512+0000 7f5d7dde6980 20 searching for the correct realm 2024-03-04T18:23:41.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.520+0000 7f5d7c0ff640 0 -- 172.21.15.47:0/2560560516 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7f5d640e2060 msgr2=0x7f5d640c28a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.524+0000 7f5d7b8fe640 0 -- 172.21.15.47:0/2560560516 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f5d7406b070 msgr2=0x7f5d740781f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.532+0000 7f5d7dde6980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.532+0000 7f5d7dde6980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.532+0000 7f5d7dde6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.532+0000 7f5d7dde6980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.532+0000 7f5d7dde6980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.532+0000 7f5d7dde6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.532+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.532+0000 7f5d7dde6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.532+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 zone default found 2024-03-04T18:23:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 4 Realm: () 2024-03-04T18:23:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 zonegroup default 2024-03-04T18:23:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 started sync module instance, tier type = 2024-03-04T18:23:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.536+0000 7f5d7dde6980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.556+0000 7f5d7dde6980 20 add_watcher() i=0 2024-03-04T18:23:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.556+0000 7f5d7dde6980 20 add_watcher() i=1 2024-03-04T18:23:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.556+0000 7f5d7dde6980 20 add_watcher() i=2 2024-03-04T18:23:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.556+0000 7f5d7dde6980 20 add_watcher() i=3 2024-03-04T18:23:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.556+0000 7f5d7dde6980 20 add_watcher() i=4 2024-03-04T18:23:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.556+0000 7f5d7dde6980 20 add_watcher() i=5 2024-03-04T18:23:41.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.556+0000 7f5d7dde6980 20 add_watcher() i=6 2024-03-04T18:23:41.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.556+0000 7f5d7dde6980 20 add_watcher() i=7 2024-03-04T18:23:41.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.556+0000 7f5d7dde6980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5c31fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5d7dde6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5d7dde6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5d7dde6980 5 note: GC not initialized 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5c30fe1640 20 reqs_thread_entry: start 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5d7dde6980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5d7dde6980 20 Filter name: none 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5baa7fc640 20 reqs_thread_entry: start 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5d7dde6980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-03-04T18:23:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.560+0000 7f5d7dde6980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.564+0000 7f5d7dde6980 20 rados_obj.operate() r=0 bl.length=515 2024-03-04T18:23:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.564+0000 7f5d7dde6980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x13 2024-03-04T18:23:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.564+0000 7f5d7dde6980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:23:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.564+0000 7f5d7dde6980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:23:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.568+0000 7f5d7dde6980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-03-04T18:23:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.568+0000 7f5d7dde6980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:23:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.568+0000 7f5d7dde6980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-03-04T18:23:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.568+0000 7f5c5affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529869 cookie 94194694819888 notifier 7214 bl.length()=729 2024-03-04T18:23:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.568+0000 7f5c5affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-03-04T18:23:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.568+0000 7f5c5affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:23:41.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.572+0000 7f5c31fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:41.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.576+0000 7f5d7dde6980 20 remove_watcher() i=0 2024-03-04T18:23:41.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.576+0000 7f5d7dde6980 2 removed watcher, disabling cache 2024-03-04T18:23:41.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.580+0000 7f5d7dde6980 20 remove_watcher() i=1 2024-03-04T18:23:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.580+0000 7f5d7dde6980 20 remove_watcher() i=2 2024-03-04T18:23:41.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.584+0000 7f5d7dde6980 20 remove_watcher() i=3 2024-03-04T18:23:41.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.584+0000 7f5d7dde6980 20 remove_watcher() i=4 2024-03-04T18:23:41.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.588+0000 7f5d7dde6980 20 remove_watcher() i=5 2024-03-04T18:23:41.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.588+0000 7f5d7dde6980 20 remove_watcher() i=6 2024-03-04T18:23:41.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.592+0000 7f5d7dde6980 20 remove_watcher() i=7 2024-03-04T18:23:41.612 DEBUG:tasks.s3tests:Creating user foobar.client.0 on client.0 2024-03-04T18:23:41.612 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foobar.client.0 --display-name 'Mr. foobar.client.0' --email foobar.client.0+test@test.test --caps 'user-policy=*' --access-key BECDFWIJZHNVVEPFODCU --secret AVbNEMskYCMu7tEDmYjCOcTFrVoHkPIDFcIWO/EYb/pJhogJDXYgIA== --cluster ceph 2024-03-04T18:23:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:41.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.712+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 realm 2024-03-04T18:23:41.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:41.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:41.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.716+0000 7f9f41009980 20 searching for the correct realm 2024-03-04T18:23:41.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.732+0000 7f9f3eb21640 0 -- 172.21.15.47:0/2179963588 >> [v2:172.21.15.47:6808/450877494,v1:172.21.15.47:6809/450877494] conn(0x56344cf607c0 msgr2=0x56344cf62c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:23:41.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.748+0000 7f9f3f322640 0 -- 172.21.15.47:0/2179963588 >> [v2:172.21.15.152:6818/159740591,v1:172.21.15.152:6821/159740591] conn(0x7f9f300a4880 msgr2=0x7f9f30002070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:41.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:41.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:41.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:41.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:41.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:41.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:41.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:41.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:41.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 zone default found 2024-03-04T18:23:41.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 4 Realm: () 2024-03-04T18:23:41.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:41.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:41.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:41.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 zonegroup default 2024-03-04T18:23:41.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.752+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.756+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.756+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.756+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.756+0000 7f9f41009980 20 started sync module instance, tier type = 2024-03-04T18:23:41.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.756+0000 7f9f41009980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.776+0000 7f9f41009980 20 add_watcher() i=0 2024-03-04T18:23:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.776+0000 7f9f41009980 20 add_watcher() i=1 2024-03-04T18:23:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.776+0000 7f9f41009980 20 add_watcher() i=2 2024-03-04T18:23:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.776+0000 7f9f41009980 20 add_watcher() i=3 2024-03-04T18:23:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.776+0000 7f9f41009980 20 add_watcher() i=4 2024-03-04T18:23:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.776+0000 7f9f41009980 20 add_watcher() i=5 2024-03-04T18:23:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.776+0000 7f9f41009980 20 add_watcher() i=6 2024-03-04T18:23:41.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.776+0000 7f9f41009980 20 add_watcher() i=7 2024-03-04T18:23:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.776+0000 7f9f41009980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:41.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9df57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 5 note: GC not initialized 2024-03-04T18:23:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9d6ffff640 20 reqs_thread_entry: start 2024-03-04T18:23:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 20 Filter name: none 2024-03-04T18:23:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9d6dffb640 20 reqs_thread_entry: start 2024-03-04T18:23:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-03-04T18:23:41.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x0 2024-03-04T18:23:41.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:41.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 10 cache get: name=default.rgw.meta+users.email+foobar.client.0+test@test.test : miss 2024-03-04T18:23:41.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.780+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 cache put: name=default.rgw.meta+users.email+foobar.client.0+test@test.test info.flags=0x0 2024-03-04T18:23:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 adding default.rgw.meta+users.email+foobar.client.0+test@test.test to cache LRU end 2024-03-04T18:23:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 cache get: name=default.rgw.meta+users.keys+BECDFWIJZHNVVEPFODCU : miss 2024-03-04T18:23:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 cache put: name=default.rgw.meta+users.keys+BECDFWIJZHNVVEPFODCU info.flags=0x0 2024-03-04T18:23:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 adding default.rgw.meta+users.keys+BECDFWIJZHNVVEPFODCU to cache LRU end 2024-03-04T18:23:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 cache get: name=default.rgw.meta+users.keys+BECDFWIJZHNVVEPFODCU : hit (negative entry) 2024-03-04T18:23:41.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 cache get: name=default.rgw.meta+users.keys+BECDFWIJZHNVVEPFODCU : hit (negative entry) 2024-03-04T18:23:41.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:23:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.784+0000 7f9f41009980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-03-04T18:23:41.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.788+0000 7f9e227fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497166 cookie 94782629530064 notifier 7710 bl.length()=656 2024-03-04T18:23:41.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.788+0000 7f9e227fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:23:41.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.788+0000 7f9e227fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:41.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.792+0000 7f9f41009980 10 cache put: name=default.rgw.meta+users.email+foobar.client.0+test@test.test info.flags=0x7 2024-03-04T18:23:41.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.792+0000 7f9f41009980 10 moving default.rgw.meta+users.email+foobar.client.0+test@test.test to cache LRU end 2024-03-04T18:23:41.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.792+0000 7f9f41009980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:foobar.client.0+test@test.test, ofs0, ns] 2024-03-04T18:23:41.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.792+0000 7f9e227fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497167 cookie 94782629522112 notifier 7710 bl.length()=200 2024-03-04T18:23:41.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.792+0000 7f9e227fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foobar.client.0+test@test.test info.flags=0x7 2024-03-04T18:23:41.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.792+0000 7f9e227fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+foobar.client.0+test@test.test to cache LRU end 2024-03-04T18:23:41.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.796+0000 7f9f41009980 10 cache put: name=default.rgw.meta+users.keys+BECDFWIJZHNVVEPFODCU info.flags=0x7 2024-03-04T18:23:41.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.796+0000 7f9f41009980 10 moving default.rgw.meta+users.keys+BECDFWIJZHNVVEPFODCU to cache LRU end 2024-03-04T18:23:41.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.796+0000 7f9f41009980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:BECDFWIJZHNVVEPFODCU, ofs0, ns] 2024-03-04T18:23:41.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.796+0000 7f9e21ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497212 cookie 94782629486480 notifier 7710 bl.length()=189 2024-03-04T18:23:41.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.796+0000 7f9e21ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+BECDFWIJZHNVVEPFODCU info.flags=0x7 2024-03-04T18:23:41.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.796+0000 7f9e21ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+BECDFWIJZHNVVEPFODCU to cache LRU end 2024-03-04T18:23:41.802 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-04T18:23:41.802 INFO:teuthology.orchestra.run.smithi047.stdout: "user_id": "foobar.client.0", 2024-03-04T18:23:41.802 INFO:teuthology.orchestra.run.smithi047.stdout: "display_name": "Mr. foobar.client.0", 2024-03-04T18:23:41.802 INFO:teuthology.orchestra.run.smithi047.stdout: "email": "foobar.client.0+test@test.test", 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "suspended": 0, 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "max_buckets": 1000, 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "subusers": [], 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "keys": [ 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "user": "foobar.client.0", 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "access_key": "BECDFWIJZHNVVEPFODCU", 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "secret_key": "AVbNEMskYCMu7tEDmYjCOcTFrVoHkPIDFcIWO/EYb/pJhogJDXYgIA==", 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "active": true 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "swift_keys": [], 2024-03-04T18:23:41.803 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": [ 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "user-policy", 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "perm": "*" 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "default_placement": "", 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "default_storage_class": "", 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "placement_tags": [], 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "bucket_quota": { 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:41.804 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "user_quota": { 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "temp_url_keys": [], 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "rgw", 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: "mfa_ids": [] 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-04T18:23:41.805 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.800+0000 7f9df57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:41.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.804+0000 7f9f41009980 20 remove_watcher() i=0 2024-03-04T18:23:41.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.804+0000 7f9f41009980 2 removed watcher, disabling cache 2024-03-04T18:23:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.808+0000 7f9f41009980 20 remove_watcher() i=1 2024-03-04T18:23:41.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.808+0000 7f9f41009980 20 remove_watcher() i=2 2024-03-04T18:23:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.812+0000 7f9f41009980 20 remove_watcher() i=3 2024-03-04T18:23:41.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.812+0000 7f9f41009980 20 remove_watcher() i=4 2024-03-04T18:23:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.816+0000 7f9f41009980 20 remove_watcher() i=5 2024-03-04T18:23:41.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.816+0000 7f9f41009980 20 remove_watcher() i=6 2024-03-04T18:23:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.820+0000 7f9f41009980 20 remove_watcher() i=7 2024-03-04T18:23:41.840 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foobar.client.0 --totp-serial 1454427117 --totp-seed VR5FX7VZYDO5PBQMPJ32OTTW6EQCRVEYR42ZWCX6JP56AJ2JDSZPYCFEIJ2KHASK --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-03-04T18:23:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.960+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 realm 2024-03-04T18:23:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.964+0000 7f5e1cb1e980 20 searching for the correct realm 2024-03-04T18:23:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.972+0000 7f5e19e35640 0 -- 172.21.15.47:0/2088889800 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7f5e10083350 msgr2=0x7f5e1001ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:41.980+0000 7f5e1a636640 0 -- 172.21.15.47:0/2088889800 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7f5e10083350 msgr2=0x7f5ce407bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:42.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.196+0000 7f5e1cb1e980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:42.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.196+0000 7f5e1cb1e980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:42.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.196+0000 7f5e1cb1e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:42.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.196+0000 7f5e1cb1e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:42.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.196+0000 7f5e1cb1e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:42.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.196+0000 7f5e1cb1e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:42.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.196+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:42.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:42.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 zone default found 2024-03-04T18:23:42.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 4 Realm: () 2024-03-04T18:23:42.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:42.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:42.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:42.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 zonegroup default 2024-03-04T18:23:42.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.200+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.204+0000 7f5e1cb1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.204+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.204+0000 7f5e1cb1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.204+0000 7f5e1cb1e980 20 started sync module instance, tier type = 2024-03-04T18:23:42.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.204+0000 7f5e1cb1e980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.224+0000 7f5e1cb1e980 20 add_watcher() i=0 2024-03-04T18:23:42.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.224+0000 7f5e1cb1e980 20 add_watcher() i=1 2024-03-04T18:23:42.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.224+0000 7f5e1cb1e980 20 add_watcher() i=2 2024-03-04T18:23:42.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.224+0000 7f5e1cb1e980 20 add_watcher() i=3 2024-03-04T18:23:42.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.224+0000 7f5e1cb1e980 20 add_watcher() i=4 2024-03-04T18:23:42.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.224+0000 7f5e1cb1e980 20 add_watcher() i=5 2024-03-04T18:23:42.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.224+0000 7f5e1cb1e980 20 add_watcher() i=6 2024-03-04T18:23:42.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.228+0000 7f5e1cb1e980 20 add_watcher() i=7 2024-03-04T18:23:42.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.228+0000 7f5e1cb1e980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:42.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.232+0000 7f5cd0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:42.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.232+0000 7f5e1cb1e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:42.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.232+0000 7f5e1cb1e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:42.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.232+0000 7f5e1cb1e980 5 note: GC not initialized 2024-03-04T18:23:42.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.232+0000 7f5c4b7fe640 20 reqs_thread_entry: start 2024-03-04T18:23:42.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.236+0000 7f5e1cb1e980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:42.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.236+0000 7f5e1cb1e980 20 Filter name: none 2024-03-04T18:23:42.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.236+0000 7f5c497fa640 20 reqs_thread_entry: start 2024-03-04T18:23:42.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.236+0000 7f5e1cb1e980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-03-04T18:23:42.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.236+0000 7f5e1cb1e980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.236+0000 7f5e1cb1e980 20 rados_obj.operate() r=0 bl.length=468 2024-03-04T18:23:42.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.236+0000 7f5e1cb1e980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-03-04T18:23:42.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.236+0000 7f5e1cb1e980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:42.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.236+0000 7f5e1cb1e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:23:42.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.244+0000 7f5e1cb1e980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:23:42.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.244+0000 7f5e1cb1e980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:42.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.244+0000 7f5e1cb1e980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-03-04T18:23:42.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.244+0000 7f5cfdffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497168 cookie 94831487859760 notifier 7247 bl.length()=670 2024-03-04T18:23:42.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.244+0000 7f5cfdffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:23:42.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.244+0000 7f5cfdffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:42.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.248+0000 7f5cd0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:42.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.252+0000 7f5e1cb1e980 20 remove_watcher() i=0 2024-03-04T18:23:42.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.252+0000 7f5e1cb1e980 2 removed watcher, disabling cache 2024-03-04T18:23:42.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.256+0000 7f5e1cb1e980 20 remove_watcher() i=1 2024-03-04T18:23:42.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.260+0000 7f5e1cb1e980 20 remove_watcher() i=2 2024-03-04T18:23:42.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.260+0000 7f5e1cb1e980 20 remove_watcher() i=3 2024-03-04T18:23:42.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.264+0000 7f5e1cb1e980 20 remove_watcher() i=4 2024-03-04T18:23:42.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.264+0000 7f5e1cb1e980 20 remove_watcher() i=5 2024-03-04T18:23:42.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.268+0000 7f5e1cb1e980 20 remove_watcher() i=6 2024-03-04T18:23:42.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.272+0000 7f5e1cb1e980 20 remove_watcher() i=7 2024-03-04T18:23:42.292 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'roles=*' --cluster ceph 2024-03-04T18:23:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.408+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 20 realm 2024-03-04T18:23:42.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:42.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:42.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.412+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.416+0000 7fb77fd9c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.416+0000 7fb77fd9c980 20 searching for the correct realm 2024-03-04T18:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.424+0000 7fb77e0b5640 0 -- 172.21.15.47:0/2098945980 >> [v2:172.21.15.47:6824/2725445163,v1:172.21.15.47:6825/2725445163] conn(0x7fb774111040 msgr2=0x7fb77411d1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.428+0000 7fb77d8b4640 0 -- 172.21.15.47:0/2098945980 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7fb770041040 msgr2=0x7fb7700294b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:42.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:42.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 zone default found 2024-03-04T18:23:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 4 Realm: () 2024-03-04T18:23:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 zonegroup default 2024-03-04T18:23:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 started sync module instance, tier type = 2024-03-04T18:23:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.440+0000 7fb77fd9c980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.460+0000 7fb77fd9c980 20 add_watcher() i=0 2024-03-04T18:23:42.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.460+0000 7fb77fd9c980 20 add_watcher() i=1 2024-03-04T18:23:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.460+0000 7fb77fd9c980 20 add_watcher() i=2 2024-03-04T18:23:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.460+0000 7fb77fd9c980 20 add_watcher() i=3 2024-03-04T18:23:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.460+0000 7fb77fd9c980 20 add_watcher() i=4 2024-03-04T18:23:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.460+0000 7fb77fd9c980 20 add_watcher() i=5 2024-03-04T18:23:42.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.460+0000 7fb77fd9c980 20 add_watcher() i=6 2024-03-04T18:23:42.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.460+0000 7fb77fd9c980 20 add_watcher() i=7 2024-03-04T18:23:42.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.460+0000 7fb77fd9c980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb62f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb77fd9c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb77fd9c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb77fd9c980 5 note: GC not initialized 2024-03-04T18:23:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb62e7fc640 20 reqs_thread_entry: start 2024-03-04T18:23:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb77fd9c980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:42.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb77fd9c980 20 Filter name: none 2024-03-04T18:23:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb59ffff640 20 reqs_thread_entry: start 2024-03-04T18:23:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb77fd9c980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-03-04T18:23:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.464+0000 7fb77fd9c980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.468+0000 7fb77fd9c980 20 rados_obj.operate() r=0 bl.length=482 2024-03-04T18:23:42.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.468+0000 7fb77fd9c980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-03-04T18:23:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.468+0000 7fb77fd9c980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:42.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.468+0000 7fb77fd9c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:23:42.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.472+0000 7fb77fd9c980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:23:42.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.472+0000 7fb77fd9c980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:42.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.472+0000 7fb77fd9c980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-03-04T18:23:42.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.472+0000 7fb6617fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497169 cookie 94653752560224 notifier 7265 bl.length()=683 2024-03-04T18:23:42.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.472+0000 7fb6617fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:23:42.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.472+0000 7fb6617fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:42.478 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-04T18:23:42.478 INFO:teuthology.orchestra.run.smithi047.stdout: "user_id": "foobar.client.0", 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "display_name": "Mr. foobar.client.0", 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "email": "foobar.client.0+test@test.test", 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "suspended": 0, 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "max_buckets": 1000, 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "subusers": [], 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "keys": [ 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "user": "foobar.client.0", 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "access_key": "BECDFWIJZHNVVEPFODCU", 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "secret_key": "AVbNEMskYCMu7tEDmYjCOcTFrVoHkPIDFcIWO/EYb/pJhogJDXYgIA==", 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "active": true 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:42.479 INFO:teuthology.orchestra.run.smithi047.stdout: "swift_keys": [], 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": [ 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "roles", 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: "perm": "*" 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "user-policy", 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: "perm": "*" 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: "default_placement": "", 2024-03-04T18:23:42.480 INFO:teuthology.orchestra.run.smithi047.stdout: "default_storage_class": "", 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "placement_tags": [], 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "bucket_quota": { 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "user_quota": { 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:42.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:42.482 INFO:teuthology.orchestra.run.smithi047.stdout: "temp_url_keys": [], 2024-03-04T18:23:42.482 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "rgw", 2024-03-04T18:23:42.482 INFO:teuthology.orchestra.run.smithi047.stdout: "mfa_ids": [ 2024-03-04T18:23:42.482 INFO:teuthology.orchestra.run.smithi047.stdout: "1454427117" 2024-03-04T18:23:42.482 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-04T18:23:42.482 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-04T18:23:42.482 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.476+0000 7fb62f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:42.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.480+0000 7fb77fd9c980 20 remove_watcher() i=0 2024-03-04T18:23:42.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.480+0000 7fb77fd9c980 2 removed watcher, disabling cache 2024-03-04T18:23:42.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.480+0000 7fb77fd9c980 20 remove_watcher() i=1 2024-03-04T18:23:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.484+0000 7fb77fd9c980 20 remove_watcher() i=2 2024-03-04T18:23:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.484+0000 7fb77fd9c980 20 remove_watcher() i=3 2024-03-04T18:23:42.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.488+0000 7fb77fd9c980 20 remove_watcher() i=4 2024-03-04T18:23:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.488+0000 7fb77fd9c980 20 remove_watcher() i=5 2024-03-04T18:23:42.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.492+0000 7fb77fd9c980 20 remove_watcher() i=6 2024-03-04T18:23:42.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.496+0000 7fb77fd9c980 20 remove_watcher() i=7 2024-03-04T18:23:42.514 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'oidc-provider=*' --cluster ceph 2024-03-04T18:23:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:23:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:23:42.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.624+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.624+0000 7efe8766a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.624+0000 7efe8766a980 20 realm 2024-03-04T18:23:42.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.624+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.624+0000 7efe8766a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.624+0000 7efe8766a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:23:42.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.624+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.628+0000 7efe8766a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.628+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.628+0000 7efe8766a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:42.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.628+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.628+0000 7efe8766a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:23:42.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.628+0000 7efe8766a980 20 searching for the correct realm 2024-03-04T18:23:42.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.636+0000 7efe84981640 0 -- 172.21.15.47:0/85751239 >> [v2:172.21.15.152:6816/3505984861,v1:172.21.15.152:6817/3505984861] conn(0x7efe700aa3d0 msgr2=0x7efe700aa850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:42.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.640+0000 7efe85182640 0 -- 172.21.15.47:0/85751239 >> [v2:172.21.15.47:6816/851640392,v1:172.21.15.47:6817/851640392] conn(0x7efe80008040 msgr2=0x7efe8000a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:23:42.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 RGWRados::pool_iterate: got zonegroup_info.315fe362-70d7-4122-adf4-920f1212ccbd 2024-03-04T18:23:42.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 RGWRados::pool_iterate: got zone_info.797b67a4-b876-4d04-b122-10afd858d465 2024-03-04T18:23:42.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:23:42.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:23:42.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:23:42.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:23:42.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:23:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.648+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:23:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 zone default found 2024-03-04T18:23:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 4 Realm: () 2024-03-04T18:23:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 4 ZoneGroup: default (315fe362-70d7-4122-adf4-920f1212ccbd) 2024-03-04T18:23:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 4 Zone: default (797b67a4-b876-4d04-b122-10afd858d465) 2024-03-04T18:23:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:23:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 zonegroup default 2024-03-04T18:23:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:23:42.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 started sync module instance, tier type = 2024-03-04T18:23:42.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.652+0000 7efe8766a980 20 started zone id=797b67a4-b876-4d04-b122-10afd858d465 (name=default) with tier type = 2024-03-04T18:23:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.672+0000 7efe8766a980 20 add_watcher() i=0 2024-03-04T18:23:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.672+0000 7efe8766a980 20 add_watcher() i=1 2024-03-04T18:23:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.672+0000 7efe8766a980 20 add_watcher() i=2 2024-03-04T18:23:42.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.672+0000 7efe8766a980 20 add_watcher() i=3 2024-03-04T18:23:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.672+0000 7efe8766a980 20 add_watcher() i=4 2024-03-04T18:23:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.672+0000 7efe8766a980 20 add_watcher() i=5 2024-03-04T18:23:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.672+0000 7efe8766a980 20 add_watcher() i=6 2024-03-04T18:23:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.672+0000 7efe8766a980 20 add_watcher() i=7 2024-03-04T18:23:42.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.672+0000 7efe8766a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:23:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efd3bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efe8766a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:23:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efe8766a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:23:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efe8766a980 5 note: GC not initialized 2024-03-04T18:23:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efd3afe5640 20 reqs_thread_entry: start 2024-03-04T18:23:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efe8766a980 20 init_complete bucket index max shards: 11 2024-03-04T18:23:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efe8766a980 20 Filter name: none 2024-03-04T18:23:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efd38fe1640 20 reqs_thread_entry: start 2024-03-04T18:23:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efe8766a980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-03-04T18:23:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.676+0000 7efe8766a980 20 rados->read ofs=0 len=0 2024-03-04T18:23:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.680+0000 7efe8766a980 20 rados_obj.operate() r=0 bl.length=495 2024-03-04T18:23:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.680+0000 7efe8766a980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-03-04T18:23:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.680+0000 7efe8766a980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.680+0000 7efe8766a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:23:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.680+0000 7efe8766a980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:23:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.680+0000 7efe8766a980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.680+0000 7efe8766a980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-03-04T18:23:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.684+0000 7efd68ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497170 cookie 94918336263184 notifier 7761 bl.length()=704 2024-03-04T18:23:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.684+0000 7efd68ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:23:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.684+0000 7efd68ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: "user_id": "foobar.client.0", 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: "display_name": "Mr. foobar.client.0", 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: "email": "foobar.client.0+test@test.test", 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: "suspended": 0, 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: "max_buckets": 1000, 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: "subusers": [], 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: "keys": [ 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: "user": "foobar.client.0", 2024-03-04T18:23:42.690 INFO:teuthology.orchestra.run.smithi047.stdout: "access_key": "BECDFWIJZHNVVEPFODCU", 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: "secret_key": "AVbNEMskYCMu7tEDmYjCOcTFrVoHkPIDFcIWO/EYb/pJhogJDXYgIA==", 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: "active": true 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: "swift_keys": [], 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": [ 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "oidc-provider", 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: "perm": "*" 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "roles", 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: "perm": "*" 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:42.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "user-policy", 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "perm": "*" 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "op_mask": "read, write, delete", 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_placement": "", 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_storage_class": "", 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "placement_tags": [], 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "bucket_quota": { 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:42.692 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "user_quota": { 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": false, 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "check_on_raw": false, 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size": -1, 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "max_size_kb": 0, 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "max_objects": -1 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "temp_url_keys": [], 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "rgw", 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "mfa_ids": [ 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: "1454427117" 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-04T18:23:42.693 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:23:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.688+0000 7efd3bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:23:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.692+0000 7efe8766a980 20 remove_watcher() i=0 2024-03-04T18:23:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.692+0000 7efe8766a980 2 removed watcher, disabling cache 2024-03-04T18:23:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.692+0000 7efe8766a980 20 remove_watcher() i=1 2024-03-04T18:23:42.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.696+0000 7efe8766a980 20 remove_watcher() i=2 2024-03-04T18:23:42.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.696+0000 7efe8766a980 20 remove_watcher() i=3 2024-03-04T18:23:42.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.700+0000 7efe8766a980 20 remove_watcher() i=4 2024-03-04T18:23:42.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.700+0000 7efe8766a980 20 remove_watcher() i=5 2024-03-04T18:23:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.704+0000 7efe8766a980 20 remove_watcher() i=6 2024-03-04T18:23:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-04T18:23:42.704+0000 7efe8766a980 20 remove_watcher() i=7 2024-03-04T18:23:42.726 INFO:tasks.s3tests:Configuring s3-tests... 2024-03-04T18:23:42.726 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T18:23:42.726 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2024-03-04T18:23:42.772 INFO:tasks.s3tests:Configuring boto... 2024-03-04T18:23:42.772 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-04T18:23:42.772 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/boto-client.0.cfg 2024-03-04T18:23:42.820 DEBUG:teuthology.orchestra.run.smithi047:s3 tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-03-04T18:23:42.984 INFO:teuthology.orchestra.run.smithi047.stdout:GLOB sdist-make: /home/ubuntu/cephtest/s3-tests-client.0/setup.py 2024-03-04T18:23:43.183 INFO:teuthology.orchestra.run.smithi047.stdout:py create: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py 2024-03-04T18:23:43.477 INFO:teuthology.orchestra.run.smithi047.stdout:py installdeps: -rrequirements.txt 2024-03-04T18:25:27.321 INFO:teuthology.orchestra.run.smithi047.stdout:py inst: /home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip 2024-03-04T18:25:29.365 INFO:teuthology.orchestra.run.smithi047.stdout:py installed: boto==2.49.0,boto3==1.24.96,botocore==1.27.96,cachetools==5.3.3,certifi==2024.2.2,chardet==5.2.0,charset-normalizer==3.3.2,colorama==0.4.6,distlib==0.3.8,exceptiongroup==1.2.0,filelock==3.13.1,gevent==24.2.1,greenlet==3.0.3,httplib2==0.22.0,idna==3.6,iniconfig==2.0.0,isodate==0.6.1,jmespath==1.0.1,lxml==5.1.0,munch==4.0.0,packaging==23.2,platformdirs==4.2.0,pluggy==1.4.0,pyparsing==3.1.1,pyproject-api==1.6.1,pytest==8.0.2,python-dateutil==2.9.0.post0,pytz==2024.1,PyYAML==6.0.1,requests==2.31.0,s3tests @ file:///home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip#sha256=abf41b81ab6f06dd76736be21e90076c985e80fa0ca95f3ba381e6f9a281d2e1,s3transfer==0.6.2,six==1.16.0,tomli==2.0.1,tox==4.13.0,urllib3==1.26.18,virtualenv==20.25.1,zope.event==5.0,zope.interface==6.2 2024-03-04T18:25:29.365 INFO:teuthology.orchestra.run.smithi047.stdout:py run-test-pre: PYTHONHASHSEED='4262744758' 2024-03-04T18:25:29.366 INFO:teuthology.orchestra.run.smithi047.stdout:py run-test: commands[0] | pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-03-04T18:25:29.548 INFO:teuthology.orchestra.run.smithi047.stdout:============================= test session starts ============================== 2024-03-04T18:25:29.548 INFO:teuthology.orchestra.run.smithi047.stdout:platform linux -- Python 3.10.6, pytest-8.0.2, pluggy-1.4.0 -- /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/python 2024-03-04T18:25:29.548 INFO:teuthology.orchestra.run.smithi047.stdout:cachedir: .tox/py/.pytest_cache 2024-03-04T18:25:29.549 INFO:teuthology.orchestra.run.smithi047.stdout:rootdir: /home/ubuntu/cephtest/s3-tests-client.0 2024-03-04T18:25:29.549 INFO:teuthology.orchestra.run.smithi047.stdout:configfile: pytest.ini 2024-03-04T18:25:31.148 INFO:teuthology.orchestra.run.smithi047.stdout:collecting ... collected 787 items / 124 deselected / 663 selected 2024-03-04T18:25:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-04T18:25:32.535 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_none PASSED [ 0%] 2024-03-04T18:25:33.428 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_empty PASSED [ 0%] 2024-03-04T18:25:34.292 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_date_and_amz_date PASSED [ 0%] 2024-03-04T18:25:35.164 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_amz_date_and_no_date PASSED [ 0%] 2024-03-04T18:25:36.042 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_none PASSED [ 0%] 2024-03-04T18:25:36.757 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_contentlength_none PASSED [ 0%] 2024-03-04T18:25:37.779 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_acl_create_contentlength_none PASSED [ 1%] 2024-03-04T18:25:38.500 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_contentlength_none PASSED [ 1%] 2024-03-04T18:25:38.698 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_empty PASSED [ 1%] 2024-03-04T18:25:38.898 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_none PASSED [ 1%] 2024-03-04T18:25:39.765 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws2 PASSED [ 1%] 2024-03-04T18:25:40.646 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws2 PASSED [ 1%] 2024-03-04T18:25:41.500 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws2 PASSED [ 1%] 2024-03-04T18:25:42.359 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws2 PASSED [ 2%] 2024-03-04T18:25:42.547 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws2 PASSED [ 2%] 2024-03-04T18:25:42.731 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws2 PASSED [ 2%] 2024-03-04T18:25:42.865 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws4 SKIPPED [ 2%] 2024-03-04T18:25:42.989 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws4 SKIPPED [ 2%] 2024-03-04T18:25:43.126 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws4 SKIPPED [ 2%] 2024-03-04T18:25:43.258 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws4 SKIPPED [ 3%] 2024-03-04T18:25:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_empty_aws4 SKIPPED [ 3%] 2024-03-04T18:25:43.522 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_none_aws4 SKIPPED [ 3%] 2024-03-04T18:25:43.674 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_invalid_aws4 SKIPPED [ 3%] 2024-03-04T18:25:43.829 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_invalid_aws4 SKIPPED [ 3%] 2024-03-04T18:25:43.959 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_empty_aws4 SKIPPED [ 3%] 2024-03-04T18:25:44.091 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_empty_aws4 SKIPPED [ 3%] 2024-03-04T18:25:44.221 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws4 SKIPPED [ 4%] 2024-03-04T18:25:44.352 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_none_aws4 SKIPPED [ 4%] 2024-03-04T18:25:44.486 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_today_aws4 SKIPPED [ 4%] 2024-03-04T18:25:44.618 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_today_aws4 SKIPPED [ 4%] 2024-03-04T18:25:44.748 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_today_aws4 SKIPPED [ 4%] 2024-03-04T18:25:44.882 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_today_aws4 SKIPPED [ 4%] 2024-03-04T18:25:45.015 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws4 SKIPPED [ 4%] 2024-03-04T18:25:45.148 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 5%] 2024-03-04T18:25:45.282 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_end_aws4 SKIPPED [ 5%] 2024-03-04T18:25:45.418 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_end_aws4 SKIPPED [ 5%] 2024-03-04T18:25:45.551 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_custom_header_aws4 SKIPPED [ 5%] 2024-03-04T18:25:45.697 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_header_aws4 SKIPPED [ 5%] 2024-03-04T18:25:45.826 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws4 SKIPPED [ 5%] 2024-03-04T18:25:45.958 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws4 SKIPPED [ 6%] 2024-03-04T18:25:46.091 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_none_aws4 SKIPPED [ 6%] 2024-03-04T18:25:46.239 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws4 SKIPPED [ 6%] 2024-03-04T18:25:46.369 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_invalid_aws4 SKIPPED [ 6%] 2024-03-04T18:25:46.501 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_empty_aws4 SKIPPED [ 6%] 2024-03-04T18:25:46.630 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_empty_aws4 SKIPPED [ 6%] 2024-03-04T18:25:46.761 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws4 SKIPPED [ 6%] 2024-03-04T18:25:46.892 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_none_aws4 SKIPPED [ 7%] 2024-03-04T18:25:47.023 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws4 SKIPPED [ 7%] 2024-03-04T18:25:47.157 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_today_aws4 SKIPPED [ 7%] 2024-03-04T18:25:47.286 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws4 SKIPPED [ 7%] 2024-03-04T18:25:47.418 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_after_today_aws4 SKIPPED [ 7%] 2024-03-04T18:25:47.548 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws4 SKIPPED [ 7%] 2024-03-04T18:25:47.685 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 7%] 2024-03-04T18:25:49.419 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_versioning_obj_read_not_exist_null PASSED [ 8%] 2024-03-04T18:26:08.562 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_object_storage_class PASSED [ 8%] 2024-03-04T18:26:28.798 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_object_storage_class_multipart PASSED [ 8%] 2024-03-04T18:26:59.033 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class PASSED [ 8%] 2024-03-04T18:27:37.892 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class_multipart PASSED [ 8%] 2024-03-04T18:27:48.466 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy PASSED [ 8%] 2024-03-04T18:28:00.623 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy_multipart PASSED [ 9%] 2024-03-04T18:28:04.290 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_atomic_dual_conditional_write_1mb PASSED [ 9%] 2024-03-04T18:28:05.174 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_atomic_write_bucket_gone PASSED [ 9%] 2024-03-04T18:28:10.276 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_1 PASSED [ 9%] 2024-03-04T18:28:15.704 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_2 PASSED [ 9%] 2024-03-04T18:28:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_bucket_policy_set_condition_operator_end_with_IfExists PASSED [ 9%] 2024-03-04T18:28:20.118 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_request_obj_tag PASSED [ 9%] 2024-03-04T18:28:21.131 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_nonexistant_bucket_rgw SKIPPED [ 10%] 2024-03-04T18:28:21.134 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_public_index SKIPPED [ 10%] 2024-03-04T18:28:21.142 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_public_index SKIPPED [ 10%] 2024-03-04T18:28:21.144 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty SKIPPED [ 10%] 2024-03-04T18:28:21.146 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty SKIPPED [ 10%] 2024-03-04T18:28:21.149 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index SKIPPED [ 10%] 2024-03-04T18:28:21.152 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index SKIPPED [ 11%] 2024-03-04T18:28:21.154 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_missingerrordoc SKIPPED [ 11%] 2024-03-04T18:28:21.157 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_missingerrordoc SKIPPED [ 11%] 2024-03-04T18:28:21.161 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_missingerrordoc SKIPPED [ 11%] 2024-03-04T18:28:21.163 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_missingerrordoc SKIPPED [ 11%] 2024-03-04T18:28:21.166 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_blockederrordoc SKIPPED [ 11%] 2024-03-04T18:28:21.168 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_pubilc_errordoc SKIPPED [ 11%] 2024-03-04T18:28:21.171 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_blockederrordoc SKIPPED [ 12%] 2024-03-04T18:28:21.173 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_blockederrordoc SKIPPED [ 12%] 2024-03-04T18:28:21.176 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_blockederrordoc SKIPPED [ 12%] 2024-03-04T18:28:21.178 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_gooderrordoc SKIPPED [ 12%] 2024-03-04T18:28:21.180 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_gooderrordoc SKIPPED [ 12%] 2024-03-04T18:28:21.182 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_gooderrordoc SKIPPED [ 12%] 2024-03-04T18:28:21.185 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_gooderrordoc SKIPPED [ 12%] 2024-03-04T18:28:21.187 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_base SKIPPED [ 13%] 2024-03-04T18:28:21.189 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path SKIPPED [ 13%] 2024-03-04T18:28:21.191 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path_upgrade SKIPPED [ 13%] 2024-03-04T18:28:21.193 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_nonwebsite SKIPPED [ 13%] 2024-03-04T18:28:21.196 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_relative SKIPPED [ 13%] 2024-03-04T18:28:21.197 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_abs SKIPPED [ 13%] 2024-03-04T18:28:21.199 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_relative SKIPPED [ 14%] 2024-03-04T18:28:21.201 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_abs SKIPPED [ 14%] 2024-03-04T18:28:21.297 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_s3_website.py::test_routing_generator XFAIL [ 14%] 2024-03-04T18:28:21.310 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests/functional/test_utils.py::test_generate PASSED [ 14%] 2024-03-04T18:28:23.719 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_short PASSED [ 14%] 2024-03-04T18:28:36.852 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_bad PASSED [ 14%] 2024-03-04T18:28:38.564 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_empty PASSED [ 14%] 2024-03-04T18:28:40.595 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_none PASSED [ 15%] 2024-03-04T18:28:42.641 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_mismatch PASSED [ 15%] 2024-03-04T18:28:44.623 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_empty PASSED [ 15%] 2024-03-04T18:28:46.684 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_none PASSED [ 15%] 2024-03-04T18:28:48.435 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contentlength_negative PASSED [ 15%] 2024-03-04T18:28:50.534 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_invalid PASSED [ 15%] 2024-03-04T18:28:52.342 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_empty PASSED [ 15%] 2024-03-04T18:28:54.141 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_none PASSED [ 16%] 2024-03-04T18:28:55.843 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_put_bad_canned_acl PASSED [ 16%] 2024-03-04T18:28:57.309 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_mismatch PASSED [ 16%] 2024-03-04T18:28:58.821 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_empty PASSED [ 16%] 2024-03-04T18:28:59.776 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_contentlength_negative PASSED [ 16%] 2024-03-04T18:29:01.459 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws2 PASSED [ 16%] 2024-03-04T18:29:03.357 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_empty_aws2 PASSED [ 17%] 2024-03-04T18:29:05.212 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_none_aws2 PASSED [ 17%] 2024-03-04T18:29:06.931 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_invalid_aws2 PASSED [ 17%] 2024-03-04T18:29:08.623 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_empty_aws2 PASSED [ 17%] 2024-03-04T18:29:10.326 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_today_aws2 PASSED [ 17%] 2024-03-04T18:29:12.000 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws2 PASSED [ 17%] 2024-03-04T18:29:13.616 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_after_end_aws2 PASSED [ 17%] 2024-03-04T18:29:15.130 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws2 PASSED [ 18%] 2024-03-04T18:29:16.743 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_none_aws2 PASSED [ 18%] 2024-03-04T18:29:17.707 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws2 PASSED [ 18%] 2024-03-04T18:29:18.676 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_empty_aws2 PASSED [ 18%] 2024-03-04T18:29:19.689 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws2 PASSED [ 18%] 2024-03-04T18:29:20.673 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws2 PASSED [ 18%] 2024-03-04T18:29:21.617 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws2 PASSED [ 19%] 2024-03-04T18:29:23.382 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy PASSED [ 19%] 2024-03-04T18:29:24.321 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_invalid_user PASSED [ 19%] 2024-03-04T18:29:25.425 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_parameter_limit PASSED [ 19%] 2024-03-04T18:29:26.699 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_put_existing_user_policy PASSED [ 19%] 2024-03-04T18:29:27.964 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy PASSED [ 19%] 2024-03-04T18:29:28.899 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy_invalid_user PASSED [ 19%] 2024-03-04T18:29:30.245 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy PASSED [ 20%] 2024-03-04T18:29:31.499 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_invalid_user PASSED [ 20%] 2024-03-04T18:29:32.998 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_from_multiple_policies PASSED [ 20%] 2024-03-04T18:29:34.188 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy PASSED [ 20%] 2024-03-04T18:29:35.394 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_user PASSED [ 20%] 2024-03-04T18:29:36.660 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_policy_name PASSED [ 20%] 2024-03-04T18:29:38.422 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_from_multiple_policies PASSED [ 20%] 2024-03-04T18:29:40.588 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_allow_bucket_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:29:42.558 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_deny_bucket_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:29:44.776 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_allow_object_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:29:47.035 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_deny_object_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:29:51.982 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_allow_multipart_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:29:56.642 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_deny_multipart_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:29:59.157 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_allow_tagging_actions_in_user_policy PASSED [ 22%] 2024-03-04T18:30:01.477 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_deny_tagging_actions_in_user_policy PASSED [ 22%] 2024-03-04T18:30:03.449 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policy_statements PASSED [ 22%] 2024-03-04T18:30:05.652 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policies PASSED [ 22%] 2024-03-04T18:30:07.147 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_iam.py::test_verify_allow_iam_actions PASSED [ 22%] 2024-03-04T18:30:09.384 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_empty PASSED [ 22%] 2024-03-04T18:30:11.914 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_distinct PASSED [ 22%] 2024-03-04T18:30:14.138 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_many PASSED [ 23%] 2024-03-04T18:30:16.469 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_many PASSED [ 23%] 2024-03-04T18:30:18.742 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_basic_key_count PASSED [ 23%] 2024-03-04T18:30:21.087 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_basic PASSED [ 23%] 2024-03-04T18:30:23.280 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_basic PASSED [ 23%] 2024-03-04T18:30:25.501 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_encoding_basic PASSED [ 23%] 2024-03-04T18:30:27.700 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_encoding_basic PASSED [ 23%] 2024-03-04T18:30:31.483 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix PASSED [ 24%] 2024-03-04T18:30:35.289 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix PASSED [ 24%] 2024-03-04T18:30:37.231 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_ends_with_delimiter PASSED [ 24%] 2024-03-04T18:30:39.176 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_ends_with_delimiter PASSED [ 24%] 2024-03-04T18:30:41.456 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_alt PASSED [ 24%] 2024-03-04T18:30:43.702 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_alt PASSED [ 24%] 2024-03-04T18:30:47.808 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_underscore PASSED [ 25%] 2024-03-04T18:30:51.765 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_underscore PASSED [ 25%] 2024-03-04T18:30:54.038 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_percentage PASSED [ 25%] 2024-03-04T18:30:56.223 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_percentage PASSED [ 25%] 2024-03-04T18:30:58.460 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_whitespace PASSED [ 25%] 2024-03-04T18:31:00.710 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_whitespace PASSED [ 25%] 2024-03-04T18:31:03.076 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_dot PASSED [ 25%] 2024-03-04T18:31:05.348 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_dot PASSED [ 26%] 2024-03-04T18:31:07.539 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_unreadable PASSED [ 26%] 2024-03-04T18:31:09.784 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_unreadable PASSED [ 26%] 2024-03-04T18:31:12.044 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_empty PASSED [ 26%] 2024-03-04T18:31:14.291 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_empty PASSED [ 26%] 2024-03-04T18:31:16.530 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_none PASSED [ 26%] 2024-03-04T18:31:18.834 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_none PASSED [ 26%] 2024-03-04T18:31:20.998 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_notempty PASSED [ 27%] 2024-03-04T18:31:23.160 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_defaultempty PASSED [ 27%] 2024-03-04T18:31:25.261 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_empty PASSED [ 27%] 2024-03-04T18:31:27.539 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_exist PASSED [ 27%] 2024-03-04T18:31:29.773 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_not_exist PASSED [ 27%] 2024-03-04T18:33:26.830 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_skip_special PASSED [ 27%] 2024-03-04T18:33:29.027 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_basic PASSED [ 28%] 2024-03-04T18:33:31.175 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_basic PASSED [ 28%] 2024-03-04T18:33:33.312 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_alt PASSED [ 28%] 2024-03-04T18:33:35.426 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_alt PASSED [ 28%] 2024-03-04T18:33:37.535 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_empty PASSED [ 28%] 2024-03-04T18:33:39.631 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_empty PASSED [ 28%] 2024-03-04T18:33:41.749 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_none PASSED [ 28%] 2024-03-04T18:33:43.923 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_none PASSED [ 29%] 2024-03-04T18:33:46.263 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_not_exist PASSED [ 29%] 2024-03-04T18:33:48.430 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_not_exist PASSED [ 29%] 2024-03-04T18:33:50.485 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_unreadable PASSED [ 29%] 2024-03-04T18:33:52.587 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_unreadable PASSED [ 29%] 2024-03-04T18:33:54.819 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_basic PASSED [ 29%] 2024-03-04T18:33:57.097 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_basic PASSED [ 30%] 2024-03-04T18:33:59.372 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_alt PASSED [ 30%] 2024-03-04T18:34:01.671 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_alt PASSED [ 30%] 2024-03-04T18:34:03.958 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_not_exist PASSED [ 30%] 2024-03-04T18:34:06.232 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_not_exist PASSED [ 30%] 2024-03-04T18:34:08.528 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_delimiter_not_exist PASSED [ 30%] 2024-03-04T18:34:10.732 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_delimiter_not_exist PASSED [ 30%] 2024-03-04T18:34:13.001 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 31%] 2024-03-04T18:34:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 31%] 2024-03-04T18:34:17.677 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_one PASSED [ 31%] 2024-03-04T18:34:20.071 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_one PASSED [ 31%] 2024-03-04T18:34:22.266 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_zero PASSED [ 31%] 2024-03-04T18:34:24.518 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_zero PASSED [ 31%] 2024-03-04T18:34:26.751 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_none PASSED [ 31%] 2024-03-04T18:34:29.006 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_none PASSED [ 32%] 2024-03-04T18:34:30.110 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_account_usage PASSED [ 32%] 2024-03-04T18:34:32.027 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_head_bucket_usage PASSED [ 32%] 2024-03-04T18:34:37.212 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_unordered PASSED [ 32%] 2024-03-04T18:34:42.436 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_unordered PASSED [ 32%] 2024-03-04T18:34:44.791 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_invalid PASSED [ 32%] 2024-03-04T18:34:47.037 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_none PASSED [ 33%] 2024-03-04T18:34:49.336 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_empty PASSED [ 33%] 2024-03-04T18:34:51.696 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken_empty PASSED [ 33%] 2024-03-04T18:34:54.095 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken PASSED [ 33%] 2024-03-04T18:34:56.438 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_both_continuationtoken_startafter PASSED [ 33%] 2024-03-04T18:34:58.653 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_unreadable PASSED [ 33%] 2024-03-04T18:35:00.888 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_unreadable PASSED [ 33%] 2024-03-04T18:35:03.064 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_not_in_list PASSED [ 34%] 2024-03-04T18:35:05.401 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_not_in_list PASSED [ 34%] 2024-03-04T18:35:07.623 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_after_list PASSED [ 34%] 2024-03-04T18:35:09.870 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_after_list PASSED [ 34%] 2024-03-04T18:35:12.543 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data PASSED [ 34%] 2024-03-04T18:35:16.032 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data_versioning PASSED [ 34%] 2024-03-04T18:35:17.939 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous PASSED [ 34%] 2024-03-04T18:35:19.919 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous PASSED [ 35%] 2024-03-04T18:35:21.639 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous_fail PASSED [ 35%] 2024-03-04T18:35:23.245 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous_fail PASSED [ 35%] 2024-03-04T18:35:24.258 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_notexist PASSED [ 35%] 2024-03-04T18:35:25.244 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_notexist PASSED [ 35%] 2024-03-04T18:35:26.206 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_notexist PASSED [ 35%] 2024-03-04T18:35:28.065 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_nonempty PASSED [ 36%] 2024-03-04T18:35:37.642 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_concurrent_set_canned_acl PASSED [ 36%] 2024-03-04T18:35:38.592 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket PASSED [ 36%] 2024-03-04T18:35:40.501 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding PASSED [ 36%] 2024-03-04T18:35:42.057 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete PASSED [ 36%] 2024-03-04T18:35:43.741 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist PASSED [ 36%] 2024-03-04T18:35:45.451 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error PASSED [ 36%] 2024-03-04T18:35:51.376 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete PASSED [ 37%] 2024-03-04T18:35:53.879 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete PASSED [ 37%] 2024-03-04T18:35:56.412 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete PASSED [ 37%] 2024-03-04T18:37:54.578 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit PASSED [ 37%] 2024-03-04T18:39:51.237 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit PASSED [ 37%] 2024-03-04T18:39:53.205 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes PASSED [ 37%] 2024-03-04T18:39:55.060 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag PASSED [ 38%] 2024-03-04T18:39:56.936 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control PASSED [ 38%] 2024-03-04T18:39:58.857 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires PASSED [ 38%] 2024-03-04T18:40:00.980 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete PASSED [ 38%] 2024-03-04T18:40:02.912 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good PASSED [ 38%] 2024-03-04T18:40:04.805 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty PASSED [ 38%] 2024-03-04T18:40:07.050 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty PASSED [ 38%] 2024-03-04T18:40:09.083 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put PASSED [ 39%] 2024-03-04T18:40:10.996 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file PASSED [ 39%] 2024-03-04T18:40:13.038 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request PASSED [ 39%] 2024-03-04T18:40:15.187 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request PASSED [ 39%] 2024-03-04T18:40:17.104 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type PASSED [ 39%] 2024-03-04T18:40:18.832 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key PASSED [ 39%] 2024-03-04T18:40:20.586 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code PASSED [ 39%] 2024-03-04T18:40:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code PASSED [ 40%] 2024-03-04T18:40:26.451 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk PASSED [ 40%] 2024-03-04T18:40:28.507 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename PASSED [ 40%] 2024-03-04T18:40:30.406 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header PASSED [ 40%] 2024-03-04T18:40:32.226 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields PASSED [ 40%] 2024-03-04T18:40:34.264 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values PASSED [ 40%] 2024-03-04T18:40:36.256 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action PASSED [ 41%] 2024-03-04T18:40:37.988 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature PASSED [ 41%] 2024-03-04T18:40:39.711 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key PASSED [ 41%] 2024-03-04T18:40:41.417 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format PASSED [ 41%] 2024-03-04T18:40:43.113 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified PASSED [ 41%] 2024-03-04T18:40:44.846 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature PASSED [ 41%] 2024-03-04T18:40:46.537 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition PASSED [ 41%] 2024-03-04T18:40:48.570 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header PASSED [ 42%] 2024-03-04T18:40:50.290 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field PASSED [ 42%] 2024-03-04T18:40:51.991 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive PASSED [ 42%] 2024-03-04T18:40:53.740 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive PASSED [ 42%] 2024-03-04T18:40:55.407 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy PASSED [ 42%] 2024-03-04T18:40:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket PASSED [ 42%] 2024-03-04T18:40:59.552 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value PASSED [ 42%] 2024-03-04T18:41:01.295 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition PASSED [ 43%] 2024-03-04T18:41:02.971 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list PASSED [ 43%] 2024-03-04T18:41:04.739 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded PASSED [ 43%] 2024-03-04T18:41:06.421 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument PASSED [ 43%] 2024-03-04T18:41:08.161 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument PASSED [ 43%] 2024-03-04T18:41:09.827 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum PASSED [ 43%] 2024-03-04T18:41:14.068 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug PASSED [ 44%] 2024-03-04T18:41:15.783 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions PASSED [ 44%] 2024-03-04T18:41:17.689 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good PASSED [ 44%] 2024-03-04T18:41:19.724 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed PASSED [ 44%] 2024-03-04T18:41:21.650 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good PASSED [ 44%] 2024-03-04T18:41:23.617 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed PASSED [ 44%] 2024-03-04T18:41:25.574 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good PASSED [ 44%] 2024-03-04T18:41:28.703 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed PASSED [ 45%] 2024-03-04T18:41:30.599 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good PASSED [ 45%] 2024-03-04T18:41:32.534 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed PASSED [ 45%] 2024-03-04T18:41:34.695 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good PASSED [ 45%] 2024-03-04T18:41:36.904 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed PASSED [ 45%] 2024-03-04T18:41:39.081 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good PASSED [ 45%] 2024-03-04T18:41:40.905 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed PASSED [ 46%] 2024-03-04T18:41:43.020 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good PASSED [ 46%] 2024-03-04T18:41:45.189 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed PASSED [ 46%] 2024-03-04T18:41:47.097 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good PASSED [ 46%] 2024-03-04T18:41:49.266 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed PASSED [ 46%] 2024-03-04T18:41:51.054 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get PASSED [ 46%] 2024-03-04T18:41:52.944 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone PASSED [ 46%] 2024-03-04T18:41:54.801 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone PASSED [ 47%] 2024-03-04T18:41:56.760 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone PASSED [ 47%] 2024-03-04T18:41:58.431 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head PASSED [ 47%] 2024-03-04T18:41:59.417 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist PASSED [ 47%] 2024-03-04T18:42:01.899 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended PASSED [ 47%] 2024-03-04T18:42:03.768 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl PASSED [ 47%] 2024-03-04T18:42:05.657 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl PASSED [ 47%] 2024-03-04T18:42:10.676 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime PASSED [ 48%] 2024-03-04T18:42:12.558 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated PASSED [ 48%] 2024-03-04T18:42:14.526 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers PASSED [ 48%] 2024-03-04T18:42:16.366 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl PASSED [ 48%] 2024-03-04T18:42:18.257 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl PASSED [ 48%] 2024-03-04T18:42:20.007 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone PASSED [ 48%] 2024-03-04T18:42:21.913 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone PASSED [ 49%] 2024-03-04T18:42:23.960 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired PASSED [ 49%] 2024-03-04T18:42:26.075 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant PASSED [ 49%] 2024-03-04T18:42:27.952 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero PASSED [ 49%] 2024-03-04T18:42:29.853 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range PASSED [ 49%] 2024-03-04T18:42:31.696 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range PASSED [ 49%] 2024-03-04T18:42:33.626 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put PASSED [ 49%] 2024-03-04T18:42:35.703 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access PASSED [ 50%] 2024-03-04T18:42:37.600 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated PASSED [ 50%] 2024-03-04T18:42:39.589 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired PASSED [ 50%] 2024-03-04T18:42:40.705 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha PASSED [ 50%] 2024-03-04T18:42:41.810 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one PASSED [ 50%] 2024-03-04T18:42:42.922 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two PASSED [ 50%] 2024-03-04T18:42:44.710 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 PASSED [ 50%] 2024-03-04T18:42:46.444 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 PASSED [ 51%] 2024-03-04T18:42:48.419 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 PASSED [ 51%] 2024-03-04T18:42:49.966 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 PASSED [ 51%] 2024-03-04T18:42:51.581 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name PASSED [ 51%] 2024-03-04T18:42:52.626 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip PASSED [ 51%] 2024-03-04T18:42:53.581 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore PASSED [ 51%] 2024-03-04T18:42:55.063 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long PASSED [ 52%] 2024-03-04T18:42:56.031 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end PASSED [ 52%] 2024-03-04T18:42:56.956 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot PASSED [ 52%] 2024-03-04T18:42:57.921 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash PASSED [ 52%] 2024-03-04T18:42:58.909 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot PASSED [ 52%] 2024-03-04T18:43:00.584 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists PASSED [ 52%] 2024-03-04T18:43:02.182 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location PASSED [ 52%] 2024-03-04T18:43:03.896 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner PASSED [ 53%] 2024-03-04T18:43:05.503 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl PASSED [ 53%] 2024-03-04T18:43:07.132 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl PASSED [ 53%] 2024-03-04T18:43:08.827 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default PASSED [ 53%] 2024-03-04T18:43:10.397 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create PASSED [ 53%] 2024-03-04T18:43:12.153 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned PASSED [ 53%] 2024-03-04T18:43:13.777 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite PASSED [ 53%] 2024-03-04T18:43:15.366 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread PASSED [ 54%] 2024-03-04T18:43:17.233 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default PASSED [ 54%] 2024-03-04T18:43:19.218 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create PASSED [ 54%] 2024-03-04T18:43:21.343 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned PASSED [ 54%] 2024-03-04T18:43:23.309 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite PASSED [ 54%] 2024-03-04T18:43:25.256 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread PASSED [ 54%] 2024-03-04T18:43:27.342 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread PASSED [ 55%] 2024-03-04T18:43:29.479 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol PASSED [ 55%] 2024-03-04T18:43:31.648 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner PASSED [ 55%] 2024-03-04T18:43:33.964 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes PASSED [ 55%] 2024-03-04T18:43:35.701 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private PASSED [ 55%] 2024-03-04T18:43:37.810 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl PASSED [ 55%] 2024-03-04T18:43:39.946 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write PASSED [ 55%] 2024-03-04T18:43:42.140 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp PASSED [ 56%] 2024-03-04T18:43:44.307 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read PASSED [ 56%] 2024-03-04T18:43:46.425 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp PASSED [ 56%] 2024-03-04T18:43:49.534 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol PASSED [ 56%] 2024-03-04T18:43:52.192 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read PASSED [ 56%] 2024-03-04T18:43:54.889 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp PASSED [ 56%] 2024-03-04T18:43:57.627 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write PASSED [ 57%] 2024-03-04T18:44:00.428 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp PASSED [ 57%] 2024-03-04T18:44:02.299 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user PASSED [ 57%] 2024-03-04T18:44:04.871 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_no_grants PASSED [ 57%] 2024-03-04T18:44:06.795 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants PASSED [ 57%] 2024-03-04T18:44:08.895 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants PASSED [ 57%] 2024-03-04T18:44:10.857 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email PASSED [ 57%] 2024-03-04T18:44:12.701 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist PASSED [ 58%] 2024-03-04T18:44:14.952 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all PASSED [ 58%] 2024-03-04T18:44:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private PASSED [ 58%] 2024-03-04T18:44:20.880 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private PASSED [ 58%] 2024-03-04T18:44:23.807 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread PASSED [ 58%] 2024-03-04T18:44:26.743 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread PASSED [ 58%] 2024-03-04T18:44:29.748 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite PASSED [ 58%] 2024-03-04T18:44:32.664 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite PASSED [ 59%] 2024-03-04T18:44:35.629 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private PASSED [ 59%] 2024-03-04T18:44:38.612 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread PASSED [ 59%] 2024-03-04T18:44:41.489 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite PASSED [ 59%] 2024-03-04T18:44:44.323 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private PASSED [ 59%] 2024-03-04T18:44:47.104 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread PASSED [ 59%] 2024-03-04T18:44:49.940 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite PASSED [ 60%] 2024-03-04T18:44:54.169 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list PASSED [ 60%] 2024-03-04T18:44:58.099 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime PASSED [ 60%] 2024-03-04T18:44:59.140 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous PASSED [ 60%] 2024-03-04T18:45:00.129 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth PASSED [ 60%] 2024-03-04T18:45:01.091 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth PASSED [ 60%] 2024-03-04T18:45:02.998 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha PASSED [ 60%] 2024-03-04T18:45:04.910 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit PASSED [ 61%] 2024-03-04T18:45:06.394 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period PASSED [ 61%] 2024-03-04T18:45:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen PASSED [ 61%] 2024-03-04T18:45:10.521 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding PASSED [ 61%] 2024-03-04T18:45:16.000 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names PASSED [ 61%] 2024-03-04T18:45:18.620 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix PASSED [ 61%] 2024-03-04T18:45:20.982 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size PASSED [ 61%] 2024-03-04T18:45:30.152 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m PASSED [ 62%] 2024-03-04T18:45:32.271 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket PASSED [ 62%] 2024-03-04T18:45:34.422 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype PASSED [ 62%] 2024-03-04T18:45:36.481 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself PASSED [ 62%] 2024-03-04T18:45:38.420 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata PASSED [ 62%] 2024-03-04T18:45:41.425 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket PASSED [ 62%] 2024-03-04T18:45:43.881 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket PASSED [ 63%] 2024-03-04T18:45:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket PASSED [ 63%] 2024-03-04T18:45:49.557 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl PASSED [ 63%] 2024-03-04T18:45:53.694 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata PASSED [ 63%] 2024-03-04T18:45:57.560 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata PASSED [ 63%] 2024-03-04T18:45:59.201 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found PASSED [ 63%] 2024-03-04T18:46:00.945 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found PASSED [ 63%] 2024-03-04T18:46:06.956 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket PASSED [ 64%] 2024-03-04T18:46:09.383 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding PASSED [ 64%] 2024-03-04T18:46:52.012 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload PASSED [ 64%] 2024-03-04T18:46:53.905 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty PASSED [ 64%] 2024-03-04T18:46:56.361 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small PASSED [ 64%] 2024-03-04T18:47:03.858 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small PASSED [ 64%] 2024-03-04T18:47:06.254 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range PASSED [ 65%] 2024-03-04T18:47:09.949 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range PASSED [ 65%] 2024-03-04T18:47:35.300 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names PASSED [ 65%] 2024-03-04T18:48:16.982 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload PASSED [ 65%] 2024-03-04T18:49:08.009 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned PASSED [ 65%] 2024-03-04T18:52:53.051 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part PASSED [ 65%] 2024-03-04T18:53:22.556 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes PASSED [ 65%] 2024-03-04T18:54:02.189 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes PASSED [ 66%] 2024-03-04T18:54:05.382 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small PASSED [ 66%] 2024-03-04T18:54:19.093 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents PASSED [ 66%] 2024-03-04T18:54:30.272 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object PASSED [ 66%] 2024-03-04T18:54:37.543 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload PASSED [ 66%] 2024-03-04T18:54:39.508 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found PASSED [ 66%] 2024-03-04T18:54:51.321 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload PASSED [ 66%] 2024-03-04T18:54:53.819 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner PASSED [ 67%] 2024-03-04T18:54:55.844 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part PASSED [ 67%] 2024-03-04T18:54:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag PASSED [ 67%] 2024-03-04T18:55:26.164 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part PASSED [ 67%] 2024-03-04T18:55:28.120 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part PASSED [ 67%] 2024-03-04T18:55:30.077 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue PASSED [ 67%] 2024-03-04T18:55:32.136 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors PASSED [ 68%] 2024-03-04T18:55:42.512 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response PASSED [ 68%] 2024-03-04T18:55:47.676 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard PASSED [ 68%] 2024-03-04T18:55:52.584 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option PASSED [ 68%] 2024-03-04T18:55:54.744 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object PASSED [ 68%] 2024-03-04T18:55:56.841 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant PASSED [ 68%] 2024-03-04T18:55:58.823 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging PASSED [ 68%] 2024-03-04T18:56:02.911 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb PASSED [ 69%] 2024-03-04T18:56:10.875 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb PASSED [ 69%] 2024-03-04T18:56:24.341 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb PASSED [ 69%] 2024-03-04T18:56:29.208 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb PASSED [ 69%] 2024-03-04T18:56:39.117 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb PASSED [ 69%] 2024-03-04T18:56:54.763 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb PASSED [ 69%] 2024-03-04T18:56:59.211 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb PASSED [ 69%] 2024-03-04T18:57:08.155 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb PASSED [ 70%] 2024-03-04T18:57:23.343 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb PASSED [ 70%] 2024-03-04T18:57:27.970 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb PASSED [ 70%] 2024-03-04T18:57:30.169 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write PASSED [ 70%] 2024-03-04T18:57:33.305 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last PASSED [ 70%] 2024-03-04T18:57:35.238 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code PASSED [ 70%] 2024-03-04T18:57:46.317 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code PASSED [ 71%] 2024-03-04T18:57:48.294 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code PASSED [ 71%] 2024-03-04T18:57:50.189 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code PASSED [ 71%] 2024-03-04T18:57:52.308 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range PASSED [ 71%] 2024-03-04T18:57:54.219 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object PASSED [ 71%] 2024-03-04T18:57:57.129 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend PASSED [ 71%] 2024-03-04T18:58:19.685 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove PASSED [ 71%] 2024-03-04T18:58:23.107 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head PASSED [ 72%] 2024-03-04T18:58:25.709 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal PASSED [ 72%] 2024-03-04T18:58:28.706 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite PASSED [ 72%] 2024-03-04T18:58:31.858 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended PASSED [ 72%] 2024-03-04T18:58:41.976 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions PASSED [ 72%] 2024-03-04T18:58:53.375 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all PASSED [ 72%] 2024-03-04T18:59:33.046 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names PASSED [ 73%] 2024-03-04T19:00:36.189 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart PASSED [ 73%] 2024-03-04T19:00:41.792 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker PASSED [ 73%] 2024-03-04T19:00:47.384 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version PASSED [ 73%] 2024-03-04T19:00:50.193 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete PASSED [ 73%] 2024-03-04T19:00:53.264 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker PASSED [ 73%] 2024-03-04T19:00:55.606 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create PASSED [ 73%] 2024-03-04T19:00:58.911 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl PASSED [ 74%] 2024-03-04T19:01:02.193 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified PASSED [ 74%] 2024-03-04T19:01:07.897 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove PASSED [ 74%] 2024-03-04T19:01:09.743 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set PASSED [ 74%] 2024-03-04T19:01:11.558 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get PASSED [ 74%] 2024-03-04T19:01:13.351 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id PASSED [ 74%] 2024-03-04T19:01:15.079 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long PASSED [ 74%] 2024-03-04T19:01:16.753 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id PASSED [ 75%] 2024-03-04T19:01:18.644 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status PASSED [ 75%] 2024-03-04T19:01:20.429 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date PASSED [ 75%] 2024-03-04T19:01:22.117 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date PASSED [ 75%] 2024-03-04T19:01:24.212 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent PASSED [ 75%] 2024-03-04T19:01:26.022 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker PASSED [ 75%] 2024-03-04T19:01:27.769 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter PASSED [ 76%] 2024-03-04T19:01:29.483 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter PASSED [ 76%] 2024-03-04T19:01:31.485 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart PASSED [ 76%] 2024-03-04T19:01:33.160 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date PASSED [ 76%] 2024-03-04T19:01:35.234 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b PASSED [ 76%] 2024-03-04T19:01:37.172 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb PASSED [ 76%] 2024-03-04T19:01:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB PASSED [ 76%] 2024-03-04T19:01:41.998 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b PASSED [ 77%] 2024-03-04T19:01:44.021 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head PASSED [ 77%] 2024-03-04T19:01:46.014 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present PASSED [ 77%] 2024-03-04T19:01:47.882 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key PASSED [ 77%] 2024-03-04T19:01:49.628 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 PASSED [ 77%] 2024-03-04T19:01:51.500 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 PASSED [ 77%] 2024-03-04T19:01:53.180 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key PASSED [ 77%] 2024-03-04T19:01:54.959 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c PASSED [ 78%] 2024-03-04T19:03:11.214 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload PASSED [ 78%] 2024-03-04T19:04:25.138 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload PASSED [ 78%] 2024-03-04T19:04:47.514 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download PASSED [ 78%] 2024-03-04T19:04:49.600 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request PASSED [ 78%] 2024-03-04T19:04:51.661 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head PASSED [ 78%] 2024-03-04T19:04:53.616 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present PASSED [ 79%] 2024-03-04T19:04:55.414 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key PASSED [ 79%] 2024-03-04T19:04:57.110 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared PASSED [ 79%] 2024-03-04T19:05:51.681 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload PASSED [ 79%] 2024-03-04T19:06:12.054 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 PASSED [ 79%] 2024-03-04T19:06:32.625 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 PASSED [ 79%] 2024-03-04T19:06:34.640 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request PASSED [ 79%] 2024-03-04T19:06:36.590 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b PASSED [ 80%] 2024-03-04T19:06:38.651 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb PASSED [ 80%] 2024-03-04T19:06:41.575 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB PASSED [ 80%] 2024-03-04T19:06:43.601 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b PASSED [ 80%] 2024-03-04T19:06:45.525 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare PASSED [ 80%] 2024-03-04T19:06:47.715 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy PASSED [ 80%] 2024-03-04T19:06:49.792 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy PASSED [ 80%] 2024-03-04T19:06:52.148 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl PASSED [ 81%] 2024-03-04T19:06:54.602 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl PASSED [ 81%] 2024-03-04T19:06:58.096 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket PASSED [ 81%] 2024-03-04T19:07:01.479 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket PASSED [ 81%] 2024-03-04T19:07:07.582 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging PASSED [ 81%] 2024-03-04T19:07:13.261 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging PASSED [ 81%] 2024-03-04T19:07:19.337 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags PASSED [ 82%] 2024-03-04T19:07:24.922 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags PASSED [ 82%] 2024-03-04T19:07:27.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T19:07:27.541 INFO:teuthology.orchestra.run.smithi047.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags 2024-03-04T19:07:27.541 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 437, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 380, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi047 was lost: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-03-04T19:07:27.543 INFO:tasks.s3tests:Cleaning up boto... 2024-03-04T19:07:27.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-04T19:07:27.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T19:07:27.996 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-03-04T19:07:28.415 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-04T19:07:28.415 DEBUG:teuthology.orchestra.run.smithi047:> rm /home/ubuntu/cephtest/boto-client.0.cfg 2024-03-04T19:07:28.461 INFO:tasks.s3tests:Removing s3-tests... 2024-03-04T19:07:28.462 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/s3-tests-client.0 2024-03-04T19:07:28.647 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 703, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 437, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 380, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi047 was lost: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-03-04T19:07:28.840 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d9973873ee624b288e713373e543729d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 703, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 437, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 380, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi047 was lost: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-03-04T19:07:28.843 DEBUG:teuthology.run_tasks:Unwinding manager s3tests 2024-03-04T19:07:28.852 DEBUG:teuthology.run_tasks:Unwinding manager s3tests-java 2024-03-04T19:07:28.893 DEBUG:tasks.s3tests_java:S3 Tests Java: END 2024-03-04T19:07:28.893 INFO:tasks.s3tests_java:S3 Tests Java: Cleaning up s3-tests-java... 2024-03-04T19:07:28.894 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/s3-tests-java 2024-03-04T19:07:28.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-04T19:07:28.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-03-04T19:07:28.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-03-04T19:07:28.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-04T19:07:28.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-03-04T19:07:28.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-03-04T19:07:28.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-03-04T19:07:28.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-03-04T19:07:28.900 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-03-04T19:07:28.900 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-03-04T19:07:28.900 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-03-04T19:07:28.900 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-03-04T19:07:28.900 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-03-04T19:07:28.900 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-03-04T19:07:28.947 INFO:tasks.s3tests_java:S3 Tests Java: Deleting S3 users... 2024-03-04T19:07:28.947 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid tester.client.0 --purge-data --cluster ceph 2024-03-04T19:07:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setuser ceph since I am not root 2024-03-04T19:07:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T19:07:33.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2024-03-04T19:07:33.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2024-03-04T19:07:33.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2024-03-04T19:07:33.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~5s 2024-03-04T19:07:33.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~5s 2024-03-04T19:07:33.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~5s 2024-03-04T19:07:33.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~5s 2024-03-04T19:07:33.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~5s 2024-03-04T19:07:33.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~5s 2024-03-04T19:07:33.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~5s 2024-03-04T19:07:33.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~5s 2024-03-04T19:07:33.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~5s 2024-03-04T19:07:33.903 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~5s 2024-03-04T19:07:33.903 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~5s 2024-03-04T19:07:38.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~10s 2024-03-04T19:07:38.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~10s 2024-03-04T19:07:38.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~10s 2024-03-04T19:07:38.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~10s 2024-03-04T19:07:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~10s 2024-03-04T19:07:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~10s 2024-03-04T19:07:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~10s 2024-03-04T19:07:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~10s 2024-03-04T19:07:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~10s 2024-03-04T19:07:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~10s 2024-03-04T19:07:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~10s 2024-03-04T19:07:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~10s 2024-03-04T19:07:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~10s 2024-03-04T19:07:38.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~10s 2024-03-04T19:07:43.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~15s 2024-03-04T19:07:43.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~15s 2024-03-04T19:07:43.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~15s 2024-03-04T19:07:43.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~15s 2024-03-04T19:07:43.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~15s 2024-03-04T19:07:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~15s 2024-03-04T19:07:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~15s 2024-03-04T19:07:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~15s 2024-03-04T19:07:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~15s 2024-03-04T19:07:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~15s 2024-03-04T19:07:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~15s 2024-03-04T19:07:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~15s 2024-03-04T19:07:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~15s 2024-03-04T19:07:43.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~15s 2024-03-04T19:07:48.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~20s 2024-03-04T19:07:48.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~20s 2024-03-04T19:07:48.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~20s 2024-03-04T19:07:48.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~20s 2024-03-04T19:07:48.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~20s 2024-03-04T19:07:48.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~20s 2024-03-04T19:07:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~20s 2024-03-04T19:07:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~20s 2024-03-04T19:07:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~20s 2024-03-04T19:07:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~20s 2024-03-04T19:07:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~20s 2024-03-04T19:07:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~20s 2024-03-04T19:07:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~20s 2024-03-04T19:07:48.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~20s 2024-03-04T19:07:53.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~25s 2024-03-04T19:07:53.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~25s 2024-03-04T19:07:53.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~25s 2024-03-04T19:07:53.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~25s 2024-03-04T19:07:53.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~25s 2024-03-04T19:07:53.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~25s 2024-03-04T19:07:53.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~25s 2024-03-04T19:07:53.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~25s 2024-03-04T19:07:53.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~25s 2024-03-04T19:07:53.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~25s 2024-03-04T19:07:53.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~25s 2024-03-04T19:07:53.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~25s 2024-03-04T19:07:53.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~25s 2024-03-04T19:07:53.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~25s 2024-03-04T19:07:58.916 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~30s 2024-03-04T19:07:58.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~30s 2024-03-04T19:07:58.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~30s 2024-03-04T19:07:58.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~30s 2024-03-04T19:07:58.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~30s 2024-03-04T19:07:58.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~30s 2024-03-04T19:07:58.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~30s 2024-03-04T19:07:58.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~30s 2024-03-04T19:07:58.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~30s 2024-03-04T19:07:58.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~30s 2024-03-04T19:07:58.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~30s 2024-03-04T19:07:58.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~30s 2024-03-04T19:07:58.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~30s 2024-03-04T19:07:58.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~30s 2024-03-04T19:08:03.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~35s 2024-03-04T19:08:03.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~35s 2024-03-04T19:08:03.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~35s 2024-03-04T19:08:03.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~35s 2024-03-04T19:08:03.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~35s 2024-03-04T19:08:03.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~35s 2024-03-04T19:08:03.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~35s 2024-03-04T19:08:03.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~35s 2024-03-04T19:08:03.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~35s 2024-03-04T19:08:03.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~35s 2024-03-04T19:08:03.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~35s 2024-03-04T19:08:03.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~35s 2024-03-04T19:08:03.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~35s 2024-03-04T19:08:03.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~35s 2024-03-04T19:08:08.922 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~40s 2024-03-04T19:08:08.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~40s 2024-03-04T19:08:08.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~40s 2024-03-04T19:08:08.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~40s 2024-03-04T19:08:08.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~40s 2024-03-04T19:08:08.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~40s 2024-03-04T19:08:08.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~40s 2024-03-04T19:08:08.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~40s 2024-03-04T19:08:08.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~40s 2024-03-04T19:08:08.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~40s 2024-03-04T19:08:08.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~40s 2024-03-04T19:08:08.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~40s 2024-03-04T19:08:08.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~40s 2024-03-04T19:08:08.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~40s 2024-03-04T19:08:13.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~45s 2024-03-04T19:08:13.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~45s 2024-03-04T19:08:13.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~45s 2024-03-04T19:08:13.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~45s 2024-03-04T19:08:13.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~45s 2024-03-04T19:08:13.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~45s 2024-03-04T19:08:13.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~45s 2024-03-04T19:08:13.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~45s 2024-03-04T19:08:13.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~45s 2024-03-04T19:08:13.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~45s 2024-03-04T19:08:13.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~45s 2024-03-04T19:08:13.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~45s 2024-03-04T19:08:13.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~45s 2024-03-04T19:08:13.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~45s 2024-03-04T19:08:18.928 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~50s 2024-03-04T19:08:18.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~50s 2024-03-04T19:08:18.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~50s 2024-03-04T19:08:18.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~50s 2024-03-04T19:08:18.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~50s 2024-03-04T19:08:18.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~50s 2024-03-04T19:08:18.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~50s 2024-03-04T19:08:18.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~50s 2024-03-04T19:08:18.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~50s 2024-03-04T19:08:18.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~50s 2024-03-04T19:08:18.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~50s 2024-03-04T19:08:18.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~50s 2024-03-04T19:08:18.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~50s 2024-03-04T19:08:18.930 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~50s 2024-03-04T19:08:23.931 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~55s 2024-03-04T19:08:23.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~55s 2024-03-04T19:08:23.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~55s 2024-03-04T19:08:23.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~55s 2024-03-04T19:08:23.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~55s 2024-03-04T19:08:23.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~55s 2024-03-04T19:08:23.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~55s 2024-03-04T19:08:23.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~55s 2024-03-04T19:08:23.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~55s 2024-03-04T19:08:23.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~55s 2024-03-04T19:08:23.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~55s 2024-03-04T19:08:23.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~55s 2024-03-04T19:08:23.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~55s 2024-03-04T19:08:23.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~55s 2024-03-04T19:08:28.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~60s 2024-03-04T19:08:28.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~60s 2024-03-04T19:08:28.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~60s 2024-03-04T19:08:28.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~60s 2024-03-04T19:08:28.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~60s 2024-03-04T19:08:28.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~60s 2024-03-04T19:08:28.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~60s 2024-03-04T19:08:28.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~60s 2024-03-04T19:08:28.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~60s 2024-03-04T19:08:28.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~60s 2024-03-04T19:08:28.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~60s 2024-03-04T19:08:28.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~60s 2024-03-04T19:08:28.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~60s 2024-03-04T19:08:28.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~60s 2024-03-04T19:08:33.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~65s 2024-03-04T19:08:33.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~65s 2024-03-04T19:08:33.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~65s 2024-03-04T19:08:33.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~65s 2024-03-04T19:08:33.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~65s 2024-03-04T19:08:33.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~65s 2024-03-04T19:08:33.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~65s 2024-03-04T19:08:33.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~65s 2024-03-04T19:08:33.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~65s 2024-03-04T19:08:33.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~65s 2024-03-04T19:08:33.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~65s 2024-03-04T19:08:33.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~65s 2024-03-04T19:08:33.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~65s 2024-03-04T19:08:33.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~65s 2024-03-04T19:08:38.940 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~70s 2024-03-04T19:08:38.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~70s 2024-03-04T19:08:38.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~70s 2024-03-04T19:08:38.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~70s 2024-03-04T19:08:38.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~70s 2024-03-04T19:08:38.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~70s 2024-03-04T19:08:38.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~70s 2024-03-04T19:08:38.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~70s 2024-03-04T19:08:38.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~70s 2024-03-04T19:08:38.942 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~70s 2024-03-04T19:08:38.942 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~70s 2024-03-04T19:08:38.942 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~70s 2024-03-04T19:08:38.942 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~70s 2024-03-04T19:08:38.942 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~70s 2024-03-04T19:08:43.943 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~75s 2024-03-04T19:08:43.944 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~75s 2024-03-04T19:08:43.944 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~75s 2024-03-04T19:08:43.944 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~75s 2024-03-04T19:08:43.944 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~75s 2024-03-04T19:08:43.944 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~75s 2024-03-04T19:08:43.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~75s 2024-03-04T19:08:43.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~75s 2024-03-04T19:08:43.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~75s 2024-03-04T19:08:43.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~75s 2024-03-04T19:08:43.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~75s 2024-03-04T19:08:43.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~75s 2024-03-04T19:08:43.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~75s 2024-03-04T19:08:43.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~75s 2024-03-04T19:08:48.947 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~80s 2024-03-04T19:08:48.947 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~80s 2024-03-04T19:08:48.947 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~80s 2024-03-04T19:08:48.947 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~80s 2024-03-04T19:08:48.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~80s 2024-03-04T19:08:48.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~80s 2024-03-04T19:08:48.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~80s 2024-03-04T19:08:48.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~80s 2024-03-04T19:08:48.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~80s 2024-03-04T19:08:48.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~80s 2024-03-04T19:08:48.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~80s 2024-03-04T19:08:48.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~80s 2024-03-04T19:08:48.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~80s 2024-03-04T19:08:48.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~80s 2024-03-04T19:08:53.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~85s 2024-03-04T19:08:53.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~85s 2024-03-04T19:08:53.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~85s 2024-03-04T19:08:53.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~85s 2024-03-04T19:08:53.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~85s 2024-03-04T19:08:53.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~85s 2024-03-04T19:08:53.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~85s 2024-03-04T19:08:53.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~85s 2024-03-04T19:08:53.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~85s 2024-03-04T19:08:53.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~85s 2024-03-04T19:08:53.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~85s 2024-03-04T19:08:53.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~85s 2024-03-04T19:08:53.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~85s 2024-03-04T19:08:53.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~85s 2024-03-04T19:08:58.952 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~90s 2024-03-04T19:08:58.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~90s 2024-03-04T19:08:58.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~90s 2024-03-04T19:08:58.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~90s 2024-03-04T19:08:58.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~90s 2024-03-04T19:08:58.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~90s 2024-03-04T19:08:58.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~90s 2024-03-04T19:08:58.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~90s 2024-03-04T19:08:58.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~90s 2024-03-04T19:08:58.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~90s 2024-03-04T19:08:58.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~90s 2024-03-04T19:08:58.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~90s 2024-03-04T19:08:58.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~90s 2024-03-04T19:08:58.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~90s 2024-03-04T19:09:03.955 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~95s 2024-03-04T19:09:03.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~95s 2024-03-04T19:09:03.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~95s 2024-03-04T19:09:03.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~95s 2024-03-04T19:09:03.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~95s 2024-03-04T19:09:03.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~95s 2024-03-04T19:09:03.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~95s 2024-03-04T19:09:03.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~95s 2024-03-04T19:09:03.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~95s 2024-03-04T19:09:03.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~95s 2024-03-04T19:09:03.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~95s 2024-03-04T19:09:03.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~95s 2024-03-04T19:09:03.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~95s 2024-03-04T19:09:03.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~95s 2024-03-04T19:09:08.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~100s 2024-03-04T19:09:08.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~100s 2024-03-04T19:09:08.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~100s 2024-03-04T19:09:08.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~100s 2024-03-04T19:09:08.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~100s 2024-03-04T19:09:08.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~100s 2024-03-04T19:09:08.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~100s 2024-03-04T19:09:08.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~100s 2024-03-04T19:09:08.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~100s 2024-03-04T19:09:08.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~100s 2024-03-04T19:09:08.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~100s 2024-03-04T19:09:08.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~100s 2024-03-04T19:09:08.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~100s 2024-03-04T19:09:08.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~100s 2024-03-04T19:09:13.961 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~105s 2024-03-04T19:09:13.961 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~105s 2024-03-04T19:09:13.961 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~105s 2024-03-04T19:09:13.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~105s 2024-03-04T19:09:13.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~105s 2024-03-04T19:09:13.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~105s 2024-03-04T19:09:13.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~105s 2024-03-04T19:09:13.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~105s 2024-03-04T19:09:13.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~105s 2024-03-04T19:09:13.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~105s 2024-03-04T19:09:13.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~105s 2024-03-04T19:09:13.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~105s 2024-03-04T19:09:13.963 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~105s 2024-03-04T19:09:13.963 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~105s 2024-03-04T19:09:18.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~110s 2024-03-04T19:09:18.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~110s 2024-03-04T19:09:18.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~110s 2024-03-04T19:09:18.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~110s 2024-03-04T19:09:18.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~110s 2024-03-04T19:09:18.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~110s 2024-03-04T19:09:18.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~110s 2024-03-04T19:09:18.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~110s 2024-03-04T19:09:18.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~110s 2024-03-04T19:09:18.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~110s 2024-03-04T19:09:18.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~110s 2024-03-04T19:09:18.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~110s 2024-03-04T19:09:18.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~110s 2024-03-04T19:09:18.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~110s 2024-03-04T19:09:23.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~115s 2024-03-04T19:09:23.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~115s 2024-03-04T19:09:23.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~115s 2024-03-04T19:09:23.968 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~115s 2024-03-04T19:09:23.968 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~115s 2024-03-04T19:09:23.968 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~115s 2024-03-04T19:09:23.968 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~115s 2024-03-04T19:09:23.968 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~115s 2024-03-04T19:09:23.968 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~115s 2024-03-04T19:09:23.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~115s 2024-03-04T19:09:23.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~115s 2024-03-04T19:09:23.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~115s 2024-03-04T19:09:23.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~115s 2024-03-04T19:09:23.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~115s 2024-03-04T19:09:28.971 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~120s 2024-03-04T19:09:28.971 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~120s 2024-03-04T19:09:28.971 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~120s 2024-03-04T19:09:28.971 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~120s 2024-03-04T19:09:28.971 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~120s 2024-03-04T19:09:28.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~120s 2024-03-04T19:09:28.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~120s 2024-03-04T19:09:28.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~120s 2024-03-04T19:09:28.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~120s 2024-03-04T19:09:28.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~120s 2024-03-04T19:09:28.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~120s 2024-03-04T19:09:28.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~120s 2024-03-04T19:09:28.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~120s 2024-03-04T19:09:28.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~120s 2024-03-04T19:09:33.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~125s 2024-03-04T19:09:33.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~125s 2024-03-04T19:09:33.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~125s 2024-03-04T19:09:33.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~125s 2024-03-04T19:09:33.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~125s 2024-03-04T19:09:33.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~125s 2024-03-04T19:09:33.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~125s 2024-03-04T19:09:33.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~125s 2024-03-04T19:09:33.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~125s 2024-03-04T19:09:33.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~125s 2024-03-04T19:09:33.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~125s 2024-03-04T19:09:33.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~125s 2024-03-04T19:09:33.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~125s 2024-03-04T19:09:33.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~125s 2024-03-04T19:09:38.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~130s 2024-03-04T19:09:38.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~130s 2024-03-04T19:09:38.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~130s 2024-03-04T19:09:38.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~130s 2024-03-04T19:09:38.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~130s 2024-03-04T19:09:38.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~130s 2024-03-04T19:09:38.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~130s 2024-03-04T19:09:38.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~130s 2024-03-04T19:09:38.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~130s 2024-03-04T19:09:38.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~130s 2024-03-04T19:09:38.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~130s 2024-03-04T19:09:38.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~130s 2024-03-04T19:09:38.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~130s 2024-03-04T19:09:38.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~130s 2024-03-04T19:09:43.979 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~135s 2024-03-04T19:09:43.979 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~135s 2024-03-04T19:09:43.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~135s 2024-03-04T19:09:43.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~135s 2024-03-04T19:09:43.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~135s 2024-03-04T19:09:43.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~135s 2024-03-04T19:09:43.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~135s 2024-03-04T19:09:43.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~135s 2024-03-04T19:09:43.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~135s 2024-03-04T19:09:43.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~135s 2024-03-04T19:09:43.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~135s 2024-03-04T19:09:43.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~135s 2024-03-04T19:09:43.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~135s 2024-03-04T19:09:43.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~135s 2024-03-04T19:09:48.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~140s 2024-03-04T19:09:48.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~140s 2024-03-04T19:09:48.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~140s 2024-03-04T19:09:48.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~140s 2024-03-04T19:09:48.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~140s 2024-03-04T19:09:53.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~145s 2024-03-04T19:09:53.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~145s 2024-03-04T19:09:53.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~145s 2024-03-04T19:09:53.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~145s 2024-03-04T19:09:53.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~145s 2024-03-04T19:09:53.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~145s 2024-03-04T19:09:53.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~145s 2024-03-04T19:09:53.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~145s 2024-03-04T19:09:53.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~145s 2024-03-04T19:09:53.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~145s 2024-03-04T19:09:53.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~145s 2024-03-04T19:09:53.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~145s 2024-03-04T19:09:53.987 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~145s 2024-03-04T19:09:53.987 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~145s 2024-03-04T19:09:58.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~150s 2024-03-04T19:09:58.989 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~150s 2024-03-04T19:09:58.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~150s 2024-03-04T19:09:58.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~150s 2024-03-04T19:09:58.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~150s 2024-03-04T19:09:58.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~150s 2024-03-04T19:09:58.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~150s 2024-03-04T19:09:58.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~150s 2024-03-04T19:09:58.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~150s 2024-03-04T19:09:58.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~150s 2024-03-04T19:09:58.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~150s 2024-03-04T19:09:58.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~150s 2024-03-04T19:09:58.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~150s 2024-03-04T19:09:58.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~150s 2024-03-04T19:10:03.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~155s 2024-03-04T19:10:03.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~155s 2024-03-04T19:10:03.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~155s 2024-03-04T19:10:03.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~155s 2024-03-04T19:10:03.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~155s 2024-03-04T19:10:03.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~155s 2024-03-04T19:10:03.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~155s 2024-03-04T19:10:03.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~155s 2024-03-04T19:10:03.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~155s 2024-03-04T19:10:03.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~155s 2024-03-04T19:10:03.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~155s 2024-03-04T19:10:03.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~155s 2024-03-04T19:10:03.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~155s 2024-03-04T19:10:03.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~155s 2024-03-04T19:10:08.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~160s 2024-03-04T19:10:08.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~160s 2024-03-04T19:10:08.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~160s 2024-03-04T19:10:08.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~160s 2024-03-04T19:10:08.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~160s 2024-03-04T19:10:08.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~160s 2024-03-04T19:10:08.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~160s 2024-03-04T19:10:08.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~160s 2024-03-04T19:10:08.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~160s 2024-03-04T19:10:08.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~160s 2024-03-04T19:10:08.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~160s 2024-03-04T19:10:08.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~160s 2024-03-04T19:10:08.997 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~160s 2024-03-04T19:10:08.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~160s 2024-03-04T19:10:13.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~165s 2024-03-04T19:10:13.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~165s 2024-03-04T19:10:13.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~165s 2024-03-04T19:10:13.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~165s 2024-03-04T19:10:13.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~165s 2024-03-04T19:10:13.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~165s 2024-03-04T19:10:13.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~165s 2024-03-04T19:10:13.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~165s 2024-03-04T19:10:13.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~165s 2024-03-04T19:10:13.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~165s 2024-03-04T19:10:13.999 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~165s 2024-03-04T19:10:14.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~165s 2024-03-04T19:10:14.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~165s 2024-03-04T19:10:14.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~165s 2024-03-04T19:10:19.001 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~170s 2024-03-04T19:10:19.001 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~170s 2024-03-04T19:10:19.001 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~170s 2024-03-04T19:10:19.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~170s 2024-03-04T19:10:19.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~170s 2024-03-04T19:10:19.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~170s 2024-03-04T19:10:19.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~170s 2024-03-04T19:10:19.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~170s 2024-03-04T19:10:19.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~170s 2024-03-04T19:10:19.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~170s 2024-03-04T19:10:19.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~170s 2024-03-04T19:10:19.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~170s 2024-03-04T19:10:19.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~170s 2024-03-04T19:10:19.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~170s 2024-03-04T19:10:24.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~175s 2024-03-04T19:10:24.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~175s 2024-03-04T19:10:24.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~175s 2024-03-04T19:10:24.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~175s 2024-03-04T19:10:24.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~175s 2024-03-04T19:10:24.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~175s 2024-03-04T19:10:24.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~175s 2024-03-04T19:10:24.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~175s 2024-03-04T19:10:24.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~175s 2024-03-04T19:10:24.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~175s 2024-03-04T19:10:24.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~175s 2024-03-04T19:10:24.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~175s 2024-03-04T19:10:24.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~175s 2024-03-04T19:10:24.005 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~175s 2024-03-04T19:10:29.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~180s 2024-03-04T19:10:29.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~180s 2024-03-04T19:10:29.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~180s 2024-03-04T19:10:29.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~180s 2024-03-04T19:10:29.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~180s 2024-03-04T19:10:29.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~180s 2024-03-04T19:10:29.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~180s 2024-03-04T19:10:29.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~180s 2024-03-04T19:10:29.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~180s 2024-03-04T19:10:29.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~180s 2024-03-04T19:10:29.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~180s 2024-03-04T19:10:29.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~180s 2024-03-04T19:10:29.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~180s 2024-03-04T19:10:29.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~180s 2024-03-04T19:10:34.011 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~185s 2024-03-04T19:10:34.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~185s 2024-03-04T19:10:34.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~185s 2024-03-04T19:10:34.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~185s 2024-03-04T19:10:34.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~185s 2024-03-04T19:10:34.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~185s 2024-03-04T19:10:34.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~185s 2024-03-04T19:10:34.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~185s 2024-03-04T19:10:34.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~185s 2024-03-04T19:10:34.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~185s 2024-03-04T19:10:34.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~185s 2024-03-04T19:10:34.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~185s 2024-03-04T19:10:34.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~185s 2024-03-04T19:10:34.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~185s 2024-03-04T19:10:39.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~190s 2024-03-04T19:10:39.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~190s 2024-03-04T19:10:39.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~190s 2024-03-04T19:10:39.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~190s 2024-03-04T19:10:39.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~190s 2024-03-04T19:10:39.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~190s 2024-03-04T19:10:39.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~190s 2024-03-04T19:10:39.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~190s 2024-03-04T19:10:39.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~190s 2024-03-04T19:10:39.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~190s 2024-03-04T19:10:39.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~190s 2024-03-04T19:10:39.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~190s 2024-03-04T19:10:39.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~190s 2024-03-04T19:10:39.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~190s 2024-03-04T19:10:44.017 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~195s 2024-03-04T19:10:44.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~195s 2024-03-04T19:10:44.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~195s 2024-03-04T19:10:44.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~195s 2024-03-04T19:10:44.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~195s 2024-03-04T19:10:44.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~195s 2024-03-04T19:10:44.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~195s 2024-03-04T19:10:44.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~195s 2024-03-04T19:10:44.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~195s 2024-03-04T19:10:44.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~195s 2024-03-04T19:10:44.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~195s 2024-03-04T19:10:44.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~195s 2024-03-04T19:10:44.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~195s 2024-03-04T19:10:44.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~195s 2024-03-04T19:10:49.020 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~200s 2024-03-04T19:10:49.021 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~200s 2024-03-04T19:10:49.021 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~200s 2024-03-04T19:10:49.021 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~200s 2024-03-04T19:10:49.021 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~200s 2024-03-04T19:10:49.021 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~200s 2024-03-04T19:10:49.021 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~200s 2024-03-04T19:10:49.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~200s 2024-03-04T19:10:49.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~200s 2024-03-04T19:10:49.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~200s 2024-03-04T19:10:49.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~200s 2024-03-04T19:10:49.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~200s 2024-03-04T19:10:49.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~200s 2024-03-04T19:10:49.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~200s 2024-03-04T19:10:54.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~205s 2024-03-04T19:10:54.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~205s 2024-03-04T19:10:54.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~205s 2024-03-04T19:10:54.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~205s 2024-03-04T19:10:54.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~205s 2024-03-04T19:10:54.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~205s 2024-03-04T19:10:54.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~205s 2024-03-04T19:10:54.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~205s 2024-03-04T19:10:54.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~205s 2024-03-04T19:10:54.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~205s 2024-03-04T19:10:54.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~205s 2024-03-04T19:10:54.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~205s 2024-03-04T19:10:54.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~205s 2024-03-04T19:10:54.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~205s 2024-03-04T19:10:59.028 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~210s 2024-03-04T19:10:59.028 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~210s 2024-03-04T19:10:59.028 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~210s 2024-03-04T19:10:59.028 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~210s 2024-03-04T19:10:59.028 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~210s 2024-03-04T19:10:59.028 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~210s 2024-03-04T19:10:59.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~210s 2024-03-04T19:10:59.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~210s 2024-03-04T19:10:59.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~210s 2024-03-04T19:10:59.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~210s 2024-03-04T19:10:59.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~210s 2024-03-04T19:10:59.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~210s 2024-03-04T19:10:59.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~210s 2024-03-04T19:10:59.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~210s 2024-03-04T19:11:04.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~215s 2024-03-04T19:11:04.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~215s 2024-03-04T19:11:04.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~215s 2024-03-04T19:11:04.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~215s 2024-03-04T19:11:04.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~215s 2024-03-04T19:11:04.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~215s 2024-03-04T19:11:04.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~215s 2024-03-04T19:11:04.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~215s 2024-03-04T19:11:04.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~215s 2024-03-04T19:11:04.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~215s 2024-03-04T19:11:04.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~215s 2024-03-04T19:11:04.033 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~215s 2024-03-04T19:11:04.033 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~215s 2024-03-04T19:11:04.033 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~215s 2024-03-04T19:11:09.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~220s 2024-03-04T19:11:09.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~220s 2024-03-04T19:11:09.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~220s 2024-03-04T19:11:09.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~220s 2024-03-04T19:11:09.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~220s 2024-03-04T19:11:09.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~220s 2024-03-04T19:11:09.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~220s 2024-03-04T19:11:09.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~220s 2024-03-04T19:11:09.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~220s 2024-03-04T19:11:09.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~220s 2024-03-04T19:11:09.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~220s 2024-03-04T19:11:09.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~220s 2024-03-04T19:11:09.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~220s 2024-03-04T19:11:09.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~220s 2024-03-04T19:11:14.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~225s 2024-03-04T19:11:14.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~225s 2024-03-04T19:11:14.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~225s 2024-03-04T19:11:14.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~225s 2024-03-04T19:11:14.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~225s 2024-03-04T19:11:14.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~225s 2024-03-04T19:11:14.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~225s 2024-03-04T19:11:14.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~225s 2024-03-04T19:11:14.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~225s 2024-03-04T19:11:14.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~225s 2024-03-04T19:11:14.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~225s 2024-03-04T19:11:14.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~225s 2024-03-04T19:11:14.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~225s 2024-03-04T19:11:14.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~225s 2024-03-04T19:11:19.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~230s 2024-03-04T19:11:19.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~230s 2024-03-04T19:11:19.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~230s 2024-03-04T19:11:19.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~230s 2024-03-04T19:11:19.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~230s 2024-03-04T19:11:19.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~230s 2024-03-04T19:11:19.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~230s 2024-03-04T19:11:19.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~230s 2024-03-04T19:11:19.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~230s 2024-03-04T19:11:19.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~230s 2024-03-04T19:11:19.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~230s 2024-03-04T19:11:19.043 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~230s 2024-03-04T19:11:19.043 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~230s 2024-03-04T19:11:19.043 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~230s 2024-03-04T19:11:24.044 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~235s 2024-03-04T19:11:24.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~235s 2024-03-04T19:11:24.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~235s 2024-03-04T19:11:24.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~235s 2024-03-04T19:11:24.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~235s 2024-03-04T19:11:24.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~235s 2024-03-04T19:11:24.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~235s 2024-03-04T19:11:24.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~235s 2024-03-04T19:11:24.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~235s 2024-03-04T19:11:24.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~235s 2024-03-04T19:11:24.046 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~235s 2024-03-04T19:11:24.046 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~235s 2024-03-04T19:11:24.046 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~235s 2024-03-04T19:11:24.046 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~235s 2024-03-04T19:11:29.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~240s 2024-03-04T19:11:29.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~240s 2024-03-04T19:11:29.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~240s 2024-03-04T19:11:29.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~240s 2024-03-04T19:11:29.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~240s 2024-03-04T19:11:29.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~240s 2024-03-04T19:11:29.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~240s 2024-03-04T19:11:29.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~240s 2024-03-04T19:11:29.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~240s 2024-03-04T19:11:29.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~240s 2024-03-04T19:11:29.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~240s 2024-03-04T19:11:29.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~240s 2024-03-04T19:11:29.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~240s 2024-03-04T19:11:29.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~240s 2024-03-04T19:11:34.050 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~245s 2024-03-04T19:11:34.050 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~245s 2024-03-04T19:11:34.050 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~245s 2024-03-04T19:11:34.050 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~245s 2024-03-04T19:11:34.050 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~245s 2024-03-04T19:11:34.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~245s 2024-03-04T19:11:34.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~245s 2024-03-04T19:11:34.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~245s 2024-03-04T19:11:34.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~245s 2024-03-04T19:11:34.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~245s 2024-03-04T19:11:34.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~245s 2024-03-04T19:11:34.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~245s 2024-03-04T19:11:34.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~245s 2024-03-04T19:11:34.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~245s 2024-03-04T19:11:39.053 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~250s 2024-03-04T19:11:39.053 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~250s 2024-03-04T19:11:39.053 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~250s 2024-03-04T19:11:39.053 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~250s 2024-03-04T19:11:39.053 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~250s 2024-03-04T19:11:39.053 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~250s 2024-03-04T19:11:39.054 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~250s 2024-03-04T19:11:39.054 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~250s 2024-03-04T19:11:39.054 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~250s 2024-03-04T19:11:39.054 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~250s 2024-03-04T19:11:39.054 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~250s 2024-03-04T19:11:39.054 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~250s 2024-03-04T19:11:39.054 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~250s 2024-03-04T19:11:39.054 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~250s 2024-03-04T19:11:44.056 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~255s 2024-03-04T19:11:44.056 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~255s 2024-03-04T19:11:44.056 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~255s 2024-03-04T19:11:44.056 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~255s 2024-03-04T19:11:44.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~255s 2024-03-04T19:11:44.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~255s 2024-03-04T19:11:44.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~255s 2024-03-04T19:11:44.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~255s 2024-03-04T19:11:44.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~255s 2024-03-04T19:11:44.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~255s 2024-03-04T19:11:44.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~255s 2024-03-04T19:11:44.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~255s 2024-03-04T19:11:44.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~255s 2024-03-04T19:11:44.058 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~255s 2024-03-04T19:11:49.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~260s 2024-03-04T19:11:49.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~260s 2024-03-04T19:11:49.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~260s 2024-03-04T19:11:49.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~260s 2024-03-04T19:11:49.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~260s 2024-03-04T19:11:49.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~260s 2024-03-04T19:11:49.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~260s 2024-03-04T19:11:49.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~260s 2024-03-04T19:11:49.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~260s 2024-03-04T19:11:49.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~260s 2024-03-04T19:11:49.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~260s 2024-03-04T19:11:49.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~260s 2024-03-04T19:11:49.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~260s 2024-03-04T19:11:49.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~260s 2024-03-04T19:11:54.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~265s 2024-03-04T19:11:54.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~265s 2024-03-04T19:11:54.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~265s 2024-03-04T19:11:54.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~265s 2024-03-04T19:11:54.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~265s 2024-03-04T19:11:54.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~265s 2024-03-04T19:11:54.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~265s 2024-03-04T19:11:54.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~265s 2024-03-04T19:11:54.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~265s 2024-03-04T19:11:54.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~265s 2024-03-04T19:11:54.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~265s 2024-03-04T19:11:54.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~265s 2024-03-04T19:11:54.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~265s 2024-03-04T19:11:54.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~265s 2024-03-04T19:11:59.064 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~270s 2024-03-04T19:11:59.064 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~270s 2024-03-04T19:11:59.064 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~270s 2024-03-04T19:11:59.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~270s 2024-03-04T19:11:59.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~270s 2024-03-04T19:11:59.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~270s 2024-03-04T19:11:59.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~270s 2024-03-04T19:11:59.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~270s 2024-03-04T19:11:59.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~270s 2024-03-04T19:11:59.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~270s 2024-03-04T19:11:59.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~270s 2024-03-04T19:11:59.066 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~270s 2024-03-04T19:11:59.066 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~270s 2024-03-04T19:11:59.066 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~270s 2024-03-04T19:12:04.067 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~275s 2024-03-04T19:12:04.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~275s 2024-03-04T19:12:04.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~275s 2024-03-04T19:12:04.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~275s 2024-03-04T19:12:04.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~275s 2024-03-04T19:12:04.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~275s 2024-03-04T19:12:04.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~275s 2024-03-04T19:12:04.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~275s 2024-03-04T19:12:04.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~275s 2024-03-04T19:12:04.069 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~275s 2024-03-04T19:12:04.069 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~275s 2024-03-04T19:12:04.069 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~275s 2024-03-04T19:12:04.069 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~275s 2024-03-04T19:12:04.069 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~275s 2024-03-04T19:12:09.070 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~280s 2024-03-04T19:12:09.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~280s 2024-03-04T19:12:09.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~280s 2024-03-04T19:12:09.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~280s 2024-03-04T19:12:09.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~280s 2024-03-04T19:12:09.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~280s 2024-03-04T19:12:09.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~280s 2024-03-04T19:12:09.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~280s 2024-03-04T19:12:09.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~280s 2024-03-04T19:12:09.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~280s 2024-03-04T19:12:09.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~280s 2024-03-04T19:12:09.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~280s 2024-03-04T19:12:09.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~280s 2024-03-04T19:12:09.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~280s 2024-03-04T19:12:14.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~285s 2024-03-04T19:12:14.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~285s 2024-03-04T19:12:14.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~285s 2024-03-04T19:12:14.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~285s 2024-03-04T19:12:14.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~285s 2024-03-04T19:12:14.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~285s 2024-03-04T19:12:14.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~285s 2024-03-04T19:12:14.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~285s 2024-03-04T19:12:14.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~285s 2024-03-04T19:12:14.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~285s 2024-03-04T19:12:14.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~285s 2024-03-04T19:12:14.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~285s 2024-03-04T19:12:14.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~285s 2024-03-04T19:12:14.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~285s 2024-03-04T19:12:19.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~290s 2024-03-04T19:12:19.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~290s 2024-03-04T19:12:19.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~290s 2024-03-04T19:12:19.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~290s 2024-03-04T19:12:19.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~290s 2024-03-04T19:12:19.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~290s 2024-03-04T19:12:19.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~290s 2024-03-04T19:12:19.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~290s 2024-03-04T19:12:19.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~290s 2024-03-04T19:12:19.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~290s 2024-03-04T19:12:19.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~290s 2024-03-04T19:12:19.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~290s 2024-03-04T19:12:19.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~290s 2024-03-04T19:12:19.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~290s 2024-03-04T19:12:24.079 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~295s 2024-03-04T19:12:24.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~295s 2024-03-04T19:12:24.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~295s 2024-03-04T19:12:24.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~295s 2024-03-04T19:12:24.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~295s 2024-03-04T19:12:24.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~295s 2024-03-04T19:12:24.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~295s 2024-03-04T19:12:24.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~295s 2024-03-04T19:12:24.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~295s 2024-03-04T19:12:24.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~295s 2024-03-04T19:12:24.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~295s 2024-03-04T19:12:24.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~295s 2024-03-04T19:12:24.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~295s 2024-03-04T19:12:24.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~295s 2024-03-04T19:12:29.082 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~300s 2024-03-04T19:12:29.082 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~300s 2024-03-04T19:12:29.082 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~300s 2024-03-04T19:12:29.082 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~300s 2024-03-04T19:12:29.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~300s 2024-03-04T19:12:29.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~300s 2024-03-04T19:12:29.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~300s 2024-03-04T19:12:29.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~300s 2024-03-04T19:12:29.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~300s 2024-03-04T19:12:29.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~300s 2024-03-04T19:12:29.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~300s 2024-03-04T19:12:29.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~300s 2024-03-04T19:12:29.084 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~300s 2024-03-04T19:12:29.084 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~300s 2024-03-04T19:12:29.084 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-03-05T05:27:30.259 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-03-05T05:27:30.285 DEBUG:teuthology.task.console_log:Killing console logger for smithi047 2024-03-05T05:27:30.286 DEBUG:teuthology.task.console_log:Killing console logger for smithi152 2024-03-05T05:27:30.287 DEBUG:teuthology.exit:Finished running handlers